METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

专利摘要:
A computer-implemented method (3100) for overlaying a graphic image in a computing device, comprising the steps of: a) providing an overlay window; b) providing at least one visible object in the overlay window, the at least one visible object comprising a bitmap with: a first plurality of pixels (P6) that are fully transparent pixels, and a second plurality of pixels (P7; P8) that are opaque pixels or semi-transparent pixels; wherein the first plurality of pixels (P6) and the second plurality of pixels (P7; P8) are interleaved; for example in a checkerboard pattern; c) configuring (3103) the overlay window in click-through mode. The object is movable in accordance with mouse movements. A computer device for performing this method. A computer-implemented method for providing an overlay window with a semi-transparent cross (3821, 3822) or a semi-transparent textured bitmap. A portable device (4500) with a semi-transparent line (4510). A display device (4604) that provides a texture overlay.
公开号:BE1026358B1
申请号:E20185848
申请日:2018-12-02
公开日:2020-01-20
发明作者:Lambert Jacobs
申请人:Inventrans Bvba；
IPC主号:

专利说明:

METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE
Domain of the invention
The present invention relates generally to the field of graphic overlay in a computer system. More specifically, the present invention relates to a computer-implemented method for providing a graphic overlay in a computer device or in a display device. The present invention also relates to a computer device configured to perform such a method, and to a computer program product for performing such a method in a computer device or in a display device. The present invention also relates to a portable computing device and to a display device.
BACKGROUND OF THE INVENTION
FIG. 1 shows a schematic block diagram of a classical computer system 100 comprising a computer device 101 (e.g. a desktop computer), with a keyboard input connected to a keyboard 102, and a mouse input connected to a mouse device 103, and a video output connected to a display or monitor or screen 104. The computer device 101 comprises a central processing unit CPU on which runs a multi-tasking operating system O / S with a graphical user interface GUI, and one or more software applications such as e.g. a text viewer or a text editor. Input from the keyboard 102 and mouse device 103 is usually handled by device drivers (device drivers) that receive information from one or more input port (s). Such computer systems and application programs are known in the art.
In the specific example of FIG. 1, the computer device 101 generates a graphic image that is displayed on the display 104 as indicated by the rectangular area 109 (shown with rounded corners and slightly offset with respect to the display edges for illustrative purposes). The graphic image 109 of FIG. 1 includes image portions related to a desktop window (visible on the left side of the screen 104), and related to an application window showing a text document (visible on the right side of the screen 104).
About two or three decades ago, when the internet was still in its infancy, many people preferred to read (or study) textual information printed on paper rather than directly from a screen, which at the time was usually a cathode ray tube. (CRT, Cathode Ray Tube) screen. At that time, most information, for example technical information, data sheets, product catalogs, manuals, computer magazines, etc., was primarily distributed in printed form and less information was available in electronic form.
BE2018 / 5848
In contrast, nowadays (in 2017) very large amounts of information are provided or made available in electronic form, and many readers prefer to read directly from a screen, instead of first printing the information on paper and then read the paper version. This change is probably due to the advent of LCD screens (stable image, high resolution), the growth and speed of the internet, the increasing amount of information that is available electronically, the growing awareness to save trees by reducing printing , the availability of electronic document management tools (eg scanning, editing, annotating, searching), the increasing size of storage devices (eg hard disk, memory stick), the ability and speed to search electronically, the trend towards a paperless office, and the arrival of portable devices (such as smartphones, tablets, etc).
Technology has evolved considerably in the last few decades. On the one hand, printing has become easier, and with the advent of laser printers and inkjet printers, printer speed and printer quality have increased considerably (compared to a matrix printer, for example). On the other hand, display quality is also considerably increased, and high-resolution LCD panels (eg with 1920 x 1080 pixels or even 3840 x 2160 at 60 Hz frame rate) are ubiquitous. Computers have become faster, with multiple processor cores operating at high speed (eg at 2 GHz), with more RAM (eg at least 4 or at least 8 Gigabytes of RAM) allowing multiple applications to run simultaneously.
The same trend is visible for portable devices, in particular smartphones: high screen resolution (for example 1920 x 1080 pixels, or 2560 x 1440 pixels, or 3840 x 2160 or 4096 x 2160 pixels are very common), multiple processor cores working at high clock speed (eg 1.5 GHz or more), large memory size (eg 4 Gigabytes of RAM and 32 Gigabytes of flash), etc. But printing from these devices is also very simple, eg using a wireless connection (eg Wi-Fi) instead of cables. eg using a wireless connection (eg Wi-Fi) instead of cables.
So people still have a choice between reading on the screen versus printing and reading on paper. However, this technological advancement may make it harder for people to read information on a screen.
While a higher image resolution usually means a sharper image, it usually also means that the font size is smaller, and that more text can be displayed on the screen. Smaller letters and more information in itself do not always make it easier to actually read that information, and can rather increase the difficulty of reading information.
A first problem encountered when reading textual information on a screen is that many people find it difficult to stay focused when it is required to read large amounts of text carefully (eg study), especially when the text is not
BE2018 / 5848 contains a lot of formatting (such as bold or underlined characters, titles, different colors, different fonts, blank lines, etc). This is, for example, the case when reading or studying published patent documents.
This first problem is recognized in the prior art, and is addressed, for example, by a tool (tool) known as LineReader, which is commercially available on the internet at the time of writing this document. This tool provides a graphical overlay window 483 with a semi-transparent (partially transparent) line 310 following the mouse pointer 399, as schematically illustrated in FIG. 3. In FIG. 3, the line 310 is colored gray because patent documents still have to be deposited in black and white, but in reality the line 310 can be colored red or blue, for example, which is more striking and easier to distinguish than the original (native) mouse pointer or mouse cursor 399 typically provided by an operating system (such as, for example, Mac O / S from Apple Inc. with headquarters in Cupertino, or Windows from MicroSoft Corporation, Redmond), especially on high-resolution screens. People can use this tool to underline textual information that they are reading on the screen. The line can increase a user's ability to stay focused and focused.
However, as far as known to the inventors, no tool is available on the market that makes it easier for a user to find, edit, or verify data in a table or worksheet, or to locate a particular cell to locate. Although there are literally hundreds of millions of people who have been using spreadsheets and worksheets daily for at least two decades, no obvious solution has been found.
Another problem encountered when reading or studying documents on the screen, or in general, when spending many hours a day on a computer screen, is that many people experience eye fatigue, especially when reading large amounts of text on a computer screen. clear background (eg when reading black text on a white background). Unfortunately, many programs (eg PDF viewers, text editors, etc.) and many web pages provide a clear, white background, probably because these programs are designed to display the information in the way it will be printed (such as the acronym WYSIWYG, which stands for what you see is what you get and that what you see is what you get means). Screens that display images with relatively large portions of black text on a white background send relatively large amounts of light energy to the eyes, which is very uncomfortable for many people. This inconvenience usually increases when people spend more hours in front of a computer screen, as the screen size increases, as the distance to the screen decreases, or combinations thereof, and is usually worse in a poorly lit room.
This problem is also recognized in the prior art, and one of the tools available in the market to address this problem is known as F.Lux, which is a
BE2018 / 5848 provides screen-filling semi-transparent graphic overlay, shown schematically in FIG. 5. This tool can also be used to dynamically change the so-called color temperature during the course of the day. Relevant to the present invention is that tools such as F.Lux can reduce the brightness of the graphic image generated by the computer device, and thereby reduce eye fatigue.
U.S. Patent No. 6,333,753, issued December 21, 2005, discloses a technique for implementing an on-demand display widget through controlled blurring (fading) initiated by user contact with a touch sensitive input device. The document describes a semi-transparent Tool Glass, optionally using a graphic accelerator. Fading in and fading out can be achieved by dynamically changing an alpha transparency value. This document is incorporated herein by reference in its entirety, especially FIG. 8 thereof, together with the accompanying description.
U.S. Patent No. 5,798,752, issued August 25, 1998, discloses a technique in which a user can move a semi-transparent overlay window containing a so-called workpiece with the non-dominant hand, and move the mouse pointer with the dominant hand.
Many people read books on an eReader device, but it's hard to stay focused. Some people use their finger to underline the text they are reading, but when the device is set aside, the location is lost. There is a need for a more practical way to stay focused.
Summary of the invention
It is an object of the present invention to provide a computer-implemented method for overlaying a graphic image generated by a computer device with an overlay image.
It is an object of certain embodiments of the present invention to provide an overlay window that includes at least one semi-transparent visible object that covers only a minority portion of the underlying graphic image, and is movable in accordance with movements of an input device connected with the computer device.
It is an object of certain embodiments of the present invention to provide such a method that is extremely suitable for assisting a user in reading textual information on a screen.
It is an object of certain embodiments of the present invention to provide such a method that is extremely suitable for assisting a user in extracting information from a table or spreadsheet, or in editing a table or spreadsheet.
BE2018 / 5848
It is an object of certain embodiments of the present invention to provide such a method wherein the overlay image further comprises at least a second semi-transparent object covering a majority portion of the underlying graphic image.
It is an object of certain embodiments of the present invention to provide such a method with at least a first and a second semi-transparent object, wherein the second semi-transparent object has a higher transparency than the first semi-transparent object.
It is an object of certain embodiments of the present invention to provide such a method in which the transparency of the first movable object is sufficiently low, so that this object can be easily distinguished from the background, and in which the transparency of the second object is sufficiently high so that the underlying textual or alphanumeric information retains good readability.
It is an object of certain embodiments of the present invention to provide such a method that further comprises at least a third semi-transparent object, which is also movable in accordance with movements of the input device.
It is an object of certain embodiments of the present invention to provide such a method in which a substantially monochrome background color of an underlying image is transformed into a background with a texture pattern, preferably without significantly impairing the legibility of the original textual information, and / or preferably with an aesthetically more attractive background.
It is also an object of the present invention to provide a computer device adapted to perform such a method.
It is also an object of the present invention to provide a computer program product for providing a graphic overlay, which computer program product, when executed on at least one processing unit of said computer device, performs said overlay method.
It is also an object of the present invention to provide a portable computing device that provides an overlay image.
It is also an object of the present invention to provide a display device that provides an overlay method.
These and other objects are achieved by a method and a computer device and a computer program product and a computer system and a portable computing device and a display device according to embodiments of the present invention.
In a first aspect, the present invention provides a computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing
BE2018 / 5848 at least one visible object in said overlay window, the at least one visible object comprising a bitmap with: a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque pixels or semi-transparent pixels; wherein the first plurality of pixels and the second plurality of pixels are alternately positioned (interleaved). c) configuring the overlay window in click-through mode.
The overlay window can be configured as an opaque window, which means that this window can have non-transparent pixels or fully transparent pixels, but no semi-transparent pixels. (The same visual effect would be obtained with a semi-transparent window with an alpha transparency of 100%).
Alternatively, the overlay window can be configured as a semi-transparent window with an alpha transparency α in the range of 1% to 99% or from 2% to 98% or from 5% to 95%, meaning that this window is completely transparent pixels or semi-transparent can have transparent pixels, but no opaque pixels.
The bitmap with the fully transparent pixels interspersed with opaque or semi-transparent pixels is referred to herein as a perforated bitmap (by analogy with a perforated plate where one can see through the openings).
By graphic image is meant the image formed by the underlying layers of associated applications. The graphic image may comprise or be composed of a layered stack of partial images, at least some of which are provided by an application window associated with a computer application, such as a text editor, a PDF document viewer, a web browser, a spreadsheet, etc. The lower image (so-called desktop image) can be generated by an operating system, the optional other images of the stack can be generated by applications such as a text editor, PDF document viewer or editor, a web browser, a spreadsheet, a drawing program, etc.
The overlay window preferably extends over substantially the entire graphics area, for example (in a Windows environment) it can extend over the so-called work area (work area), that is the entire desktop area (minus the area occupied) by a so-called taskbar (taskbar).
The concept of fully transparent pixel is well known in the art. In practice, it is usually implemented by assigning a predefined pseudo-color value to these pixels.
The bitmap may contain at least one row and / or at least one column containing at least one fully transparent pixel that is located between two opaque or between two semi-transparent pixels, preferably at least two rows and at least two columns.
BE2018 / 5848
The bitmap may contain at least one row and / or at least one column containing at least one opaque or semi-transparent pixel that is located between two fully transparent pixels, preferably at least two rows, more preferably each row.
The bitmap may contain at least one row and / or at least one column containing a plurality of odd and even-numbered sets of pixels, an odd-numbered set being composed of only one or at least one fully transparent pixel, and wherein an even-numbered set is composed of only one or at least one opaque or semi-transparent pixel.
In one embodiment, the at least one visible object comprising the bitmap occupies a majority portion of the area of the graphic image to be overlayed, e.g. at least 75%, or at least 80% or at least 85% or at least 90% of the graphic image.
Expressed in simple terms, the semi-transparent object in this embodiment can, for example, form a film layer over the graphic image which can be used to change the average color and / or the average light intensity of the graphic image.
The at least one visible object can be substantially stationary, in the sense that it does not move in accordance with movements of an input device.
In one embodiment, the method further comprises step d) providing at least a second visible object that is movable in accordance with movements of at least one pointing device, and which occupies only a minority portion of the area of the graphic image to be overlaid.
In one embodiment, the at least one visible object comprising the bitmap is movable in accordance with movements of a pointing device and occupies only a minority portion of the area of the graphic image to be overlayed, e.g. at most 25%, or at most 20% , or at most 15%, or at most 10% or at most 5%.
Expressed in simple terms, the semi-transparent object in this embodiment can form a relatively small object, such as a small line, or a large line (which extends the full width of the screen), or a small cross, or a large cross (which extends over the full width and the full height of the screen), which moves in accordance with movements of a pointing device, for example a mouse device, a trackball, a touch pad, etc. Such an object is extremely suitable for marking or underline textual information in a text document, or alphanumeric information in a worksheet.
In one embodiment, the method further comprises the steps of:
repeatedly performing the following step: f) obtaining position information (X, Y) related to a position of a mouse pointer or mouse cursor; or obtaining motion information (dx, dy) related to a motion of the at least one pointing device;
BE2018 / 5848 and repeatedly performing the following step: g) adjusting a position of the at least one object that is movable and / or of the at least one second object that is movable, using said obtained position information or using said motion information.
For example, configuring the overlay window in click-through mode can be implemented (in an MS Windows environment) using one of the following function calls, or equivalent function calls:
SetWindowLong (Handle, GWL_EXSTYLE, WS_EX_TRANSPARENT or WS_EX_LAYERED); SetWindowLong (Handle, GWL_EXSTYLE, GetWindowLong (Handle, GWL_EXSTYLE) or WS_EX_TRANSPARENT);
As an example, obtaining position information may include querying the operating system and GUI which is the position of the native mouse cursor, for example (in an MS Windows environment) using the GetCursorPos () function.
As an example, obtaining motion information may include: configuring the O / S to send raw input messages, e.g., by registering the application with the O / S to receive raw input data, e.g., in the form of input messages known as WM_INPUT messages, for example by using the WinAPI function RegisterRawinputDevices (). It is noted that applications do not automatically receive raw input messages.
Step f) is preferably performed whenever a new message arrives. Step g) may be based on a timer, for example with a period in the range of 1 ms to 100 ms, preferably in the range of 1 ms to 60 ms. Step g) therefore does not have to be performed with the same frequency as step f).
In one embodiment, the at least one object that is movable and / or the at least one second object that is movable has an elongated shape.
For example a substantially rectangular shape, for example a rectangle with sharp edges, or a rectangle with truncated edges, or a rectangle with rounded edges. Such a visible object can easily be generated or modified (size and / or color) on the fly (gradually) and is (by choosing a suitable height and width) ideal for underlining text without distracting a user.
In one embodiment, the at least one second movable object comprises a first movable element with an elongated, horizontally oriented shape, and a second movable element with an elongated, vertically oriented shape; and wherein step g) comprises adjusting a position of the first movable element and adjusting a position of the second movable element.
BE2018 / 5848
Or in simple terms: the first movable element and the second movable element form a cross. It is an advantage of overlaying with a cross that it can clearly indicate a particular cell in a spreadsheet or table.
In one embodiment the first movable element extends over a whole width of the overlay window, and the second movable element extends over a whole height of the overlay window.
It is an advantage of overlaying with a cross that extends over the entire width and height of the screen, that not only the desired cell is clearly indicated, but also the corresponding row header and column header ( column header) of the selected cell is clearly indicated.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a pseudo-random pattern.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern according to one of the following options:
i) the regular pattern is a 2x2 pattern, and exactly one of four pixels is completely transparent;
ii) the regular pattern is a 2x2 pattern and exactly two of the four pixels are completely transparent, the two fully transparent pixels preferably being diagonally opposite each other;
iii) the regular pattern is a 2x2 pattern and exactly three of the four pixels are completely transparent.
It is an advantage of using a 2x2 pattern that it is small relative to the entire screen size, and that the human eye tends to integrate the color and intensity of the four pixel values. Depending on how many pixels of the pattern are completely transparent (1 or 2 or 3 of the four), the overall intensity (or getting dark) can be roughly adjusted. If the second group of pixels is alpha-blended (English: alpha-blended value), then both the color of these pixels as well as the alpha blending value (English: alpha-blending value) can be used to fine tune the overall intensity (or getting dark) .
In one embodiment, the method further comprises the step of: repeatedly adjusting a position of the object containing the bitmap or of the bitmap such that pixels of the underlying graphic image are overlayed (covered) by a fully transparent pixel of the first plurality pixels at a first time point, and are overlayed (covered) by an opaque or semi-transparent pixel of the second plurality of pixels at a second and optionally third and fourth time points.
BE2018 / 5848
Preferably, the adjustment is made for every frame (e.g. at 60 Hz), or every two frames (e.g. at 30 Hz) or every 3 frames (e.g. at 20 Hz), or every 4 frames (e.g. at 15 Hz ). The adjustment can, for example, be as simple as moving the second object or the entire overlay window 1 pixel to the right or to the left or up or down. This can be seen as a kind of time-multiplex.
In an alternative embodiment, a second bitmap is provided that is offset from the first bitmap of the object, and the method further comprises overlaying with the first bitmap at a first time, and overlaying with the second bitmap at a second and optional third and fourth time.
In one embodiment, step b) comprises providing a bitmap wherein the second group of pixels are extracted or derived from a texture bitmap.
The texture bitmap can be a predefined bitmap or a selectable bitmap, or a modification thereof.
The texture bitmap can be much smaller than the bitmap of the at least one object in the overlay window, in which case it is repeated and / or tiled at least once and / or mirrored and / or rotated. Extracting pixel values from a smaller texture bitmap offers the advantage of saving memory. The use of non-constant values for the second group of pixels can give the user a paper-like appearance, which is more pleasant to read than a clear white plastic-like view.
In one embodiment, the overlay window is configured as a semi-transparent window with an alpha transparency (α) in the range of 1% to 99% or from 2% to 98% or from 5% to 95%, and the method further comprises the step of of: adjusting pixel values of the second plurality of pixels extracted or derived from said texture bitmap, as a function of an alpha transparency value of the overlay window.
Preferably, the adjustment is made such that a contrast level of the texture remains substantially the same, despite variations in the alpha transparency value.
In one embodiment, the pixel values comprise three color components, for example, Red, Green and Blue, and each pixel value is adjusted using a linear expression of the respective color component itself, and optionally limiting the result of that linear expression to the range of 0 to 255.
As an example, the Red component can be adjusted according to the following formula, or an equivalent formula: R: = round ((R-Ravg) * Fcontrast + newR; limited to [0..255], where R is the original red value is of a pixel extracted from the texture bitmap, and Fcontrast is a predefined or selectable contrast factor (a floating point number) .This value preferably changes as a second-order function (e.g. as the square) of the transparency level
BE2018 / 5848
T, where Τ = 100% -α, newR is a desired average value for the red components of the overlay image, which may be different from that of the stored texture bitmap.
It is an advantage if each component value (eg Red) is calculated solely as a function of the original component value (in the example Red) and not of the other component values of that pixel (in the example: Blue and Green), because this calculates time saves, and can be implemented through a one-dimensional look-up table.
In one embodiment, the graphic image is generated as a layered composition of a desktop image and one or more images associated with said one or more applications; and the semi-transparent overlay window is created by an overlay application.
The at least one processor would typically further run an operating system (O / S) with a graphical user interface (GUI) that provides only a single mouse pointer, and one or more applications selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a text editor.
According to a second aspect, the present invention provides a computer device, comprising: at least one central processing unit, and a first memory connected to the at least one central processing unit for storing therein computer-executable instructions; computer-executable instructions comprising code fragments for performing an overlay method according to the first aspect.
The computer-executable instructions may further comprise computer-executable instructions configured to generate a graphic image in response to the operating system and / or one or more applications.
The computer-executable instructions may further comprise an operating system (O / S) with a graphical user interface (GUI) that provides only a single mouse pointer, and one or more applications selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a text editor.
By computer device is not only meant a personal computer or a desktop computer or a laptop or a tablet computer, but also portable devices such as a PDA or a smartphone or an eReader device.
In one embodiment, the computer device further comprises a graphic processing unit (GPU) with alpha mixing functionality, and a second memory connected to the graphic processing unit for storing graphic information therein; and wherein the overlay code fragments are configured to use the graphics processor to perform at least alpha blending, and optionally also time multiplexing.
BE2018 / 5848
According to a third aspect, the present invention also provides a computer system comprising: a computer device according to the second aspect; at least one pointing device connected to the computer device, the pointing device being movable by a user, the computer-executable instructions being further configured to receive input data indicative of movements of the pointing device; at least one display device connected to an output of the computer device, for displaying the graphic image mixed with the overlay image.
The pointing device can be, for example, a mouse device, or a trackball or a touchpad.
Preferably, the display device has a screen resolution of at least 1920 x 1080 pixels, or at least 2560 x 1440 pixels, or at least 3840 x 2160 pixels.
The executable instructions for receiving mouse input data are usually referred to as a mouse device driver.
According to a fourth aspect, the present invention also relates to a computer program product for providing a graphic overlay, the computer program product including executable instructions which, when executed on at least one central processing unit of a computer device according to the second aspect or a computer system according to the third aspect, cause the computer device to perform a method according to the first aspect.
The invention also relates to a computer program product for performing a method according to the first aspect on a computer device according to the second aspect or a computer system according to the third aspect.
In a fifth aspect, the present invention also provides a computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing at least a first visible object in the overlay window, the first object having an elongated shape, being horizontally oriented and occupying a minority portion of the overlay window; c) providing at least one second visible object in the overlay window, the second object having an elongated shape, being vertically oriented and occupying a minority portion of the overlay window; d) configuring the overlay window in click-through mode; repeatedly performing the following step: f) obtaining position information (X, Y) related to a position of a mouse pointer or mouse cursor; or obtaining motion information (dx, dy) related to a motion of the at least one pointing device; and repeatedly performing the following step: g) adjusting a position of the first visible object and adjusting a position of the second visible object based on the obtained position information or using the motion information.
BE2018 / 5848
The first and second object together form a cross. Such an overlay is particularly useful when extracting data from a table, or when editing a spreadsheet.
In one embodiment, the first object extends over at least 80% of a width of the overlay window; and the second object extends at least 80% of a height of the overlay window.
Preferably, the overlay window extends substantially over the entire screen, or over the entire working area of the screen (in the context of a Windows environment, this means the entire area minus the area occupied by the taskbar).
Preferably, the first object has a rectangular shape with a width of at least 80% or at least 85% or at least 90% or at least 95% or equal to 100% of the width of the screen.
Preferably, the second object has a rectangular shape with a height of at least 80% or at least 85% or at least 90% or at least 95% or equal to 100% of the height of the screen.
The height of the horizontal object and the width of the vertical object are preferably equal.
In one embodiment, the method further comprises step b) of: configuring the overlay window as a semi-transparent overlay window with an alpha transparency in the range of 5% to 95% or of 10% to 90% or from 20% to 80%.
In an alternative embodiment, the method further comprises step b) of: configuring the overlay window as a non-semi-transparent window, also referred to herein as an opaque window.
In one embodiment, the first and second objects comprise a monochrome bitmap of pixels.
In one embodiment, the first and second objects include a bitmap with: a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque or that are semi-transparent pixels to be alpha blended with pixels of windows that are below the overlay window; wherein the first plurality of pixels and the second plurality of pixels are alternately positioned.
The first and second plurality of pixels can be arranged in a checkerboard pattern.
In a sixth aspect, the present invention also provides a computer-implemented method of overlaying a graphic image in a computing device, the method comprising the steps of: a) providing a semi-transparent overlay window with an alpha transparency in the range of 5% up to 95% or from 10% to 90% or from 20% to 80%; b) providing at least one bitmap in the overlay window, the bitmap at least 50% or ten
BE2018 / 5848 occupies at least 60% or at least 70% or at least 80% or at least 90% or approximately equal to 100% of the area of the overlay window, the bitmap containing a texture bitmap; c) configuring the overlay window in click-through mode.
The use of a texture bitmap transforms a shiny or glossy, eg plastic-like, white background into a matte, eg paper-like background, whereby a nice aesthetic effect can be created that makes reading a document on a screen more pleasant.
In a seventh aspect, the present invention also provides a computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing a first visible object in the form of a vertical line; c) providing a second visible object in the form of a horizontal line segment that is located either on the left side or on the right side of the vertical line; repeatedly performing the following steps: d) obtaining position information (X, Y) from a mouse pointer or mouse cursor; e) testing whether the mouse cursor or mouse pointer is on or above the vertical line, and if the result of the test is true, proceed to step f), otherwise proceed to step h); f) if the overlay window is not already configured in non-click-through mode, configuring the overlay window in non-click-through mode; g) testing whether a mouse button has been pressed, and if the result of this test is true, dragging the vertical line; and skipping steps h) and i); h) if the overlay window is not already configured in click-through mode, configuring the overlay window in click-through mode; i) adjusting a position of the horizontal line segment.
This embodiment provides a line segment that freezes when the mouse cursor moves on the right side of the vertical line, and that moves together with the mouse cursor when the mouse cursor is on the left side of the vertical line. Moreover, this embodiment makes it possible to drag the vertical line in a very intuitive way. This embodiment is particularly suitable for translators and proofreaders.
According to an eighth aspect, the present invention also provides a portable computing device comprising: a touch screen; at least one central processing unit, and a first memory connected to the at least one central processing unit for storing therein computer-executable instructions; wherein the computer-executable instructions are configured to generate a graphic image containing textual information and to display that graphic image on the touch screen; wherein the computer-executable instructions are further configured to generate a line or an elongated object (e.g., a rectangular object with sharp or rounded edges) that covers said textual information; wherein the line or elongated object contains a plurality of semi-transparent pixels with a transparency level in the range of 1% to 99% or from 2% to 98% or from 5% to 95%, or wherein the
BE2018 / 5848 whether the elongated object contains a first plurality of fully transparent pixels and a second plurality of opaque or semi-transparent pixels that are alternately positioned; wherein the computer-executable instructions are further configured to detect a contact position on the touch screen, and to adjust a position of the line or the elongated object in accordance with the detected position.
The device can be, for example, an eReader or a smartphone device.
According to a ninth aspect, the present invention also provides a display device for displaying a graphic image provided by a computer device, the display device comprising: a display panel for generating a visible image; an input port (e.g., a video input port) for receiving said graphic image data from said computer device; an input buffer for storing the graphic image received at the input port, and a texture buffer for storing a texture bitmap, and a frame buffer for storing image data to be displayed on the display panel; a processor adapted to generate the image data to be displayed on the display panel; wherein the texture bitmap comprises a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are semi-transparent pixels, preferably with a transparency level of 1% to 99% or 2% to 98% or from 5% to 95 %, the first plurality of pixels and the second plurality of pixels being arranged alternately (interleaved). wherein the processor is adapted to generate said image data to be displayed by copying image data from the input buffer in the case that the corresponding pixel of the texture bitmap is a fully transparent pixel, and by alpha blending image data from the input buffer into the case that the corresponding texture bitmap pixel is a semi-transparent pixel.
Preferably the display device is an LCD device or an LCD monitor.
In one embodiment, the first plurality of pixels and the second plurality of pixels of the bitmap are organized in a regular pattern according to one of the following options: i) the regular pattern is a 2x2 pattern, and exactly one of the four pixels is completely transparent; ii) the regular pattern is a 2x2 pattern and exactly two of the four pixels are completely transparent, the two fully transparent pixels preferably being diagonally opposite each other; iii) the regular pattern is a 2x2 pattern and exactly three of the four pixels are completely transparent.
In one embodiment, the processor is further adapted to repeatedly adjust a position of the texture bitmap such that pixels of the underlying graphic image are overlayed (covered) by a fully transparent pixel of the first plurality of pixels at a first time, and overlayed (covered) ) by a semi-transparent pixel of the second plurality of pixels at a second and optionally third and fourth time points.
BE2018 / 5848
According to a tenth aspect, the present invention also provides a display device for displaying a graphic image provided by a computer device, the display device comprising: a display panel for generating a visible image; receiving said graphic image data from the computer device; an input buffer for storing the graphic image received at the input port, and a texture buffer for storing a texture bitmap, and a frame buffer for storing image data to be displayed on the display panel; a processor adapted to generate said image data to be displayed on the display panel; wherein the processor is adapted to generate said image data as follows: i) copying computer image data from the input buffer at even times at pixel locations for which the sum of the row index and the column index is odd, and by alpha mixing of computer image data from the input buffer with corresponding pixel data from the texture bitmap at pixel locations for which the sum of the row index and the column index is even; and ii) copying computer image data from the input buffer at uneven times at pixel locations for which the sum of the row index and column index is even, and by mixing computer image data from the input buffer with corresponding pixel data from the texture bitmap at pixel locations for which the sum of the row index and the column index is odd.
Specific and preferred aspects of the invention are included in the appended independent and dependent claims. Features of the dependent claims can be combined with features of the independent claims and with features of other dependent claims as appropriate and not merely as explicitly stated in the claims.
These and other aspects of the invention will become apparent from and clarified with reference to the embodiments described below.
Brief description of the drawings
FIG. 1 shows a schematic block diagram of a classical computer system.
FIG. 2 is a schematic representation of a so-called Z-order of three windows or image planes that can be used in the computer system of FIG. 1.
FIG. 3 shows a schematic block diagram of a computer system similar to that of FIG. 1, when executing an operating system and two applications: a text editor and a specific overlay application, known in the art as LineReader, the latter showing a semi-transparent line across the text editor window, which line can be moved by the user to underline text fragments.
FIG. 4 is a schematic representation of a so-called Z-order of four windows or image planes that can be used in the computer system of FIG. 3.
BE2018 / 5848
FIG. 5 shows a schematic block diagram of a computer system similar to that of FIG. 1, when running an operating system and two applications: a text editor and a specific overlay application known in the art as F.Lux, the latter showing a semi-transparent screen-filling overlay image over the desktop window and the text editor window.
FIG. 6 is a schematic representation of a so-called Z-order of four windows or image planes that can be used in the computer system of FIG. 5.
FIG. 7 shows a schematic block diagram of an embodiment of a computer system according to the present invention. The computer system of FIG. 7 contains a specific overlay application for implementing a specific overlay method. The overlay application, when executed in the computer system of FIG. 7, shows a semi-transparent overlay comprising at least two semi-transparent elements: (i) a horizontal line or beam that extends over substantially the entire width of the screen, and (ii) a vertical line or beam that extends substantially extends over the entire height of the screen. The horizontal and vertical lines are movable in accordance with movements of the input device.
FIG. 8 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 7.
FIG. 9 shows a schematic block diagram of another computer system according to an embodiment of the present invention. The computer system of FIG. 9 contains a specific overlay application configured to overlay at least two semi-transparent elements: i) a line, and ii) a so-called perforated bitmap. The line is movable in accordance with movements of the input device.
FIG. 10 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 9.
FIG. 11 shows a variant of the computer system of FIG. 9.
FIG. 12 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 11.
FIG. 13 shows a plurality of exemplary screenshots (screen recordings). Each row shows a series of five images that correspond to an alpha transparency or alpha mix value α of 100%, 80%, 60%, 40% and 20%, respectively. Each of these images contains a text fragment that is overlayed (covered) by two lines and / or a monochrome bitmap. Two lines are used for illustrative purposes, to illustrate both highlighting and underlining of text in the same drawing.
In FIG. 13 (a), each text fragment is overlayed by two red lines with (R, G, B) = (255.0.0). The rest of the overlay window is completely transparent.
BE2018 / 5848
In FIG. 13 (b), each text fragment is overlayed by a bitmap that contains light gray pixels with (R, G, B) = (192,192,192).
In FIG. 13 (c), each text fragment is overlayed by a bitmap containing dark gray pixels with (R, G, B) = (64, 64, 64).
In FIG. 13 (d), each text fragment is overlayed by a mere combination of the two red lines as in FIG. 13 (a) and through a light gray bitmap as in FIG. 13 (b).
In FIG. 13 (e), each text fragment is overlayed by a mere combination of the two red lines as in FIG. 13 (a) and through a dark gray bitmap as in FIG. 13 (c).
In FIG. 13 (f) to FIG. 13 (i) each text fragment is overlayed by two red lines as in
FIG. 13 (a) and the remaining area is overlayed by a so-called perforated bitmap as illustrated in FIG. 12, wherein 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are light gray pixels with (R, G, B) = (192,192,192), as can be provided by overlay methods according to embodiments of the present invention.
In FIG. 13 (g), each text fragment is overlayed by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as illustrated in FIG. 12, wherein 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are dark gray pixels with (R, G, B) = (64, 64, 64) as may be provided by overlay methods according to embodiments of the present invention.
In FIG. 13 (h), each text fragment is overlayed by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as illustrated in FIG. 12, wherein 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are black pixels with (R, G, B) = (0, 0, 0) as can be provided by overlay methods according to embodiments of the present invention.
In FIG. 13 (i), each text fragment is overlayed by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as illustrated in FIG. 12, wherein 50% of the pixels are fully transparent pixels arranged in a checkerboard pattern, and the other 50% of the pixels are white pixels with (R, G, B) = (255, 255, 255) as may be provided by overlay methods according to embodiments of the present invention.
FIG. 14 (a) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels when these pixels are overlayed by a bitmap containing 100% black pixels with (R, G, B) = (0, 0, 0) ), for three different alpha values.
FIG. 14 (b) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels
BE2018 / 5848 be overlayed by a bitmap that contains 100% white pixels with (R, G, B) = (255, 255, 255), for three different alpha values.
FIG. 15 (a) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels are overlayed by a bitmap containing 50% fully transparent pixels and 50% black pixels with (R, G, B) = (0, 0, 0), arranged in a checkerboard pattern, for three different alpha values.
FIG. 15 (b) schematically illustrates what happens to text information composed of originally black text pixels and originally white background pixels, when these pixels are overlayed by a bitmap containing 50% fully transparent pixels and 50% white pixels with (R, G, B) = (255, 255, 255) arranged in a checkerboard pattern, for three different alpha values.
FIG. 16 to FIG. 19 show four examples of a larger text fragment.
FIG. 16 shows the text fragment containing black text on a white background as can be represented by a classical PDF viewer.
FIG. 17 shows the text fragment of FIG. 16, overlayed by a blue line segment containing pixels with the color (R, G, B) = (0.0.255), and with the transparency level of the overlay window set to 50%.
FIG. 18 shows the text fragment of FIG. 16, overlayed in a manner as described in
FIG. 9 and FIG. 10, in particular, using a perforated bitmap with 50% fully transparent pixels, and 50% gray pixels with color value (128, 128, 128) arranged in a checkerboard pattern.
FIG. 19 shows a schematic block diagram of another computer system according to an embodiment of the present invention, wherein the computer system comprises a specific overlay application that provides an overlay window comprising at least two semi-transparent elements: i) a line, and ii) a so-called perforated texture bitmap . The line is movable in accordance with movements of an input device.
FIG. 20 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 19.
FIG. 21 shows a schematic block diagram of another computer system according to an embodiment of the present invention, wherein the computer system includes a specific overlay application that provides an overlay window comprising at least one semi-transparent object in the form of a so-called perforated texture bitmap.
FIG. 22 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 21.
FIG. 23 shows a schematic block diagram of another computer system according to an embodiment of the present invention, wherein the computer system has a specific overlay20
BE2018 / 5848 application that provides an overlay window that contains at least one semi-transparent object in the form of a so-called texture bitmap.
FIG. 24 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 23.
FIG. 25 shows the text fragment of FIG. 16, overlayed in a manner as described in
FIG. 19 and FIG. 20, using a perforated bitmap with 50% fully transparent pixels, and 50% gray pixels obtained from a texture bitmap, chosen such that the resulting overlayed image has a background with an average color value of approximately (128, 128, 128) for comparison possible with FIG. 16 (the contrast of the texture is adjustable, and can be somewhat exaggerated for illustrative purposes).
FIG. 26 shows the text fragment of FIG. 16, overlayed in a manner as described in
FIG. 19 and FIG. 20, wherein a semi-transparent blue line with color (0.0.255) is added and using a perforated bitmap with 50% fully transparent pixels and 50% gray pixels organized in a checkerboard pattern, the gray pixels being obtained from a texture bitmap.
FIG. 27 shows the exemplary texture bitmap that was used to display the image of FIG. 25 and FIG. 26, before perforation, before optional color adjustment and before optional contrast adjustment.
FIG. 28 schematically illustrates how overlaying with a perforated bitmap that is shifted back and forth over a single pixel can be used for time multiplexing or time averaging.
FIG. 29 (a) to (c) show examples of an image fragment obtained by overlaying a graphic image containing black text on a white background, using a method of the present invention displayed on a display device with native (original) resolution of 3840 x 2160 configured to a resolution of 2560 x 1440, with a scaling factor for text and applications of 100%.
FIG. 29 (a) shows a portion of the image of FIG. 18, viewed from a relatively large distance where the human eye does not distinguish individual pixels.
FIG. 29 (b) shows the same image zoomed in by a factor of approximately 350%.
FIG. 29 (c) shows a portion of the same image, further zoomed in by a factor of about 200%.
FIG. 30 (a) to FIG. 30 (c) show substantially the same images as those of FIG. 29 (a) to
FIG. 29 (c), displayed on the same display device, again configured with a resolution of 2560 x 1440, but configured with 125% scaling for text and applications.
FIG. 30 (a) shows what the resulting image fragment looks like when viewed from a relatively large distance where the human eye does not distinguish individual pixels.
BE2018 / 5848
FIG. 30 (b) shows the same image zoomed in by a factor of approximately 350%.
FIG. 30 (c) shows the same image, further zoomed in by a factor of about 200%.
FIG. 31 shows a flow diagram of overlay methods according to embodiments of the present invention.
FIG. 32 shows a flow diagram of overlay methods according to embodiments of the present invention.
FIG. 33 shows a flow diagram of overlay methods according to embodiments of the present invention.
FIG. 34 shows a simplified high level block diagram of software components and hardware components, located in a computer system, that typically cooperate in performing a method according to the present invention.
FIG. 35 shows an exemplary user interface window as can be used by an overlay application according to the present invention.
FIG. 36 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with a specific overlay application that provides an overlay window with a movable object having a so-called perforated bitmap. The line is movable in accordance with movements of the input device.
FIG. 37 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 36.
FIG. 38 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window with a movable object in the form of a perforated cross movable in accordance with movements of the input device.
FIG. 39 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 38.
FIG. 40 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application that provides an overlay window with a movable object in the form of a horizontal line extending the entire width of the screen, and which is movable in accordance with movements of the input device.
FIG. 41 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 40.
FIG. 42 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window with a vertical line defining a left area and a right area of the screen, and with a horizontal line extending extends across the width of the left area, and that vertically
BE2018 / 5848 is movable in accordance with movements of the input device when the mouse cursor is in the left area, and which freezes when the mouse cursor is in the right area.
FIG. 43 is a schematic representation of a Z order of four windows or image planes that can be used in the computer system of FIG. 42.
FIG. 44 shows a flow diagram of another overlay method according to embodiments of the present invention.
FIG. 45 shows a schematic block diagram of a portable device such as, for example, an eReader or a smartphone adapted to display textual information overlayed by a semi-transparent line, according to an embodiment of the present invention.
FIG. 46 shows a schematic block diagram of another computer system according to an embodiment of the present invention, which can be seen as a variant of FIG. 21, wherein the computer system provides a graphic overlay image in the form of a so-called perforated texture bitmap overlayed by the display device.
FIG. 47 is a schematic representation of a Z order of three windows or image planes that can be used in the computer system of FIG. 46, and above that a fourth semi-transparent overlay window provided by the display device.
FIG. 48 shows a schematic block diagram of another computer system according to an embodiment of the present invention, which can be seen as a variant of FIG. 46, wherein the computer system provides a graphic overlay image in the form of an unperforated texture bitmap that is overlayed by the display device.
FIG. 49 is a schematic representation of a Z order of three windows or image planes that can be used in the computer system of FIG. 48, and above a fourth semi-transparent overlay window provided by the display device.
Detailed description of illustrative embodiments
The present invention will be described with respect to particular embodiments and with reference to certain drawings, however, the invention is not limited thereto but is only limited by the claims. The figures are only schematic and non-limiting. In the drawings, the size of certain elements may be exaggerated and not drawn to scale for illustrative purposes. The dimensions and the relative dimensions sometimes do not correspond to the current practical embodiment of the invention.
Furthermore, the terms first, second and the like in the description and in the claims are used to distinguish similar elements and not necessarily for describing a sequence, neither in time, nor spatially, nor in ranking, or in any other way. It is to be understood that the terms used in this way under suitable circumstances
BE2018 / 5848 are interchangeable and that the embodiments of the invention described herein are capable of operating in a different order than described or shown herein.
It is to be noted that the term comprising, as used in the claims, is not to be interpreted as being limited to the means described thereafter; this term does not exclude other elements or steps. It can therefore be interpreted as specifying the presence of the listed characteristics, values, steps or components referred to, but does not exclude the presence or addition of one or more other characteristics, values, steps or components, or groups thereof. Thus, the scope of the term a device comprising means A and B should not be limited to devices that consist only of components A and B. It means that with respect to the present invention, A and B are the only relevant components of the device.
Reference throughout this specification to "one embodiment" or "an embodiment" means that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, occurrence of the expressions "in one embodiment" or "in an embodiment" at various places throughout this specification may not necessarily all refer to the same embodiment, but may do so. Furthermore, the specific features, structures, or characteristics may be combined in any suitable manner, as would be apparent to those skilled in the art based on this disclosure, in one or more embodiments.
Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof for the purpose of streamlining disclosure and assisting in understanding one or several of the various inventive aspects. In any case, this method of disclosure should not be interpreted as a reflection of an intention that the invention requires more features than explicitly mentioned in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all the features of a single prior disclosed embodiment. Thus, the claims following the detailed description are hereby explicitly included in this detailed description, with each independent claim as a separate embodiment of the present invention.
Furthermore, while some embodiments described herein include some, but not other, features included in other embodiments, combinations of features of different embodiments are intended to be within the scope of the invention, and constitute different embodiments, as would be understood by those skilled in the art .
BE2018 / 5848 For example, in the following claims, any of the described embodiments can be used in any combination.
Numerous specific details are set forth in the description provided here. It is, however, understood that embodiments of the invention can be practiced without these specific details. In other cases, well-known methods, structures and techniques have not been shown in detail to keep this description clear.
In this document the terms alpha mixing or mixing or mixing (of images or bitmaps) are used as synonyms. In the context of graphic overlay there is a higher image (with a higher Z order) and a lower image (with a lower Z order). The mixing can be done in hardware and / or software, and is based on a parameter called alpha mixing value α, which refers to the level of mixing of the two images, where α = 0% means that the original image is not mixed with the overlay image (or that the overlay image is completely transparent), and where α = 100% means that the resulting pixel value is that of the overlay image (or that the overlay image is completely opaque) except for fully transparent pixels. The alpha-mixing value can be expressed on a scale from 0 to 255, where α = 255 corresponds to α = 100% and α = 0 corresponds to α = 0%. For example, alpha mixing can be used to fade in or fade out a graphic image. In the context of the present invention, a second parameter is also involved in mixing, designated by the term TransparentColorValue. If a pixel of the overlay image has this predefined (false color or pseudo-color) value, then this pixel is treated as completely transparent. Fully transparent pixels are typically used, for example, when displaying a rectangular image with rounded edges. Alpha blending and fully transparent pixels are both well known in the art.
When reference is made in the present invention to a color of a pixel, not only the hue is meant (e.g., red or blue or green or yellow), but also its intensity or brightness, unless otherwise specified or clear from the context.
When reference is made in the present invention to a color expressed in R, G, B values, a pixel color is meant that can be represented by a set of color components, for example by a set of (R, G, B) values representative of a red , a green and a blue color component value, each with a value in the range of 0 to 255. Pure black is represented by (0, 0, 0) and pure white is represented by (255, 255, 255). The Ren G and B values for a gray-scale pixel are the same.
By pixel intensity or brightness or luminance of a pixel represented by a set of (R, G, B) values, is meant a value L that is approximately equal to L = R * 0.3 + G * 0.6 + B * 0.1 or approximately equal to R * 3/8 + G * 5/8 + B * 1/8, or other formulas known in the art.
BE2018 / 5848
Where in the present invention reference is made to a black, gray or white pixel, it is meant a pixel having a color whose R value and G value and B value are equal when the color is expressed in R, G, B values, for example, a black pixel has RGB values (0, 0, 0), a white pixel has RGB values (255, 255, 255), assuming an 8-bit representation has been used.
In this document, the term color of a pixel or value of a pixel refers to the set of color components, for example a combination of an R, G and B value. A black or white or gray pixel has equal values for its R, G and B components, and can therefore be represented by the color values (x, x, x). For these pixels, the single number x is also called the value of the pixel.
In this document the expression with color (R, G, B) means the same as with a color that can be expressed in terms of Red, Green and Blue color components, with red = R, green = G, blue = B, each on a scale from 0 to 255.
In this document, the terms display and monitor and screen are used as synonyms, unless explicitly stated, or unless the context clearly indicates otherwise. In the context of the present invention, the display is usually a single physical device.
In this document, the terms checkerboard pattern and checkerboard pattern are used as synonyms.
In this document, the term perforated bitmap means a bitmap that contains a first plurality of fully transparent pixels and a second plurality of other pixels. In practice, the first plurality of pixels has a predefined pseudo-color value that is recognized by a mixing unit (e.g., a software mixer or a graphics processor) to treat these pixels as completely transparent. The other pixels of the bitmap may all have the same color (monochrome), or may have different colors (e.g., extracted from a texture bitmap or may form a color gradient). The first and second plurality of pixels are preferably arranged in a pattern, e.g. in a checkerboard pattern, such as illustrated in FIG. 10 and FIG. 12 and FIG. 22 and FIG. 37.
By average is meant by averaging spatially or by averaging time, or both averaging spatially as well as by averaging time.
In this document, the size, width, or height of the screen may refer to (i) the size, width, and height of the pixel matrix of the screen (not the border around the pixel matrix), or may refer to (ii) the size, width. and height of the so-called work area, which is the same area as (i) minus the area occupied by the so-called task bar (task bar). The work area is also the area that is usually taken up by an application when it is maximized.
BE2018 / 5848
In this document, the terms line or bar are used as synonyms. They can refer to a (relatively small) line segment or block or to a rectangular area that extends substantially over the full width or full height of the screen, or over the full width or full height of the work area (work area) off the screen.
In this document, the terms word processor or text editor should not be interpreted too narrowly, because many so-called word processors (such as, for example, Microsoft Word) also have drawing options. Similarly, the term spreadsheet or worksheet should not be interpreted too narrowly, because many so-called worksheet applications, such as Microsoft Excel, also have graphic options.
In this document the expression means the overlay window is configured in click-through mode (click-through mode) or the overlay window is configured in pass-through mode (pass-through mode) that the overlay window is configured in such a way that events (events) of input devices (such as eg a mouse, keyboard, touchpad, etc.) are sent by the operating system to one or more underlying application windows or to objects thereof, despite the higher Z-order of the overlay application. This applies, among other things, to events caused by a button press (button-press) or a button release (button-release) or a button click (button-click) or a scroll event of a mouse wheel, and to movements. In the context of the present invention, the overlay application typically comprises one window configured in click-through mode, but preferably also a second, substantially opaque window (optionally with fully transparent pixels), with user interface elements.
In this document, the expression the overlay window is configured in non-click mode that the overlay window is not configured in click-through mode.
In this document, the term work area refers to a portion of the pixel area of a screen, namely, the area that will be occupied by the application 705 when it is maximized. In a Microsoft Windows environment, the work area means the entire pixel area except the so-called taskbar (taskbar), which is usually located at the bottom of the screen, but which can also be located elsewhere.
In this document, the terms checkerboard pattern and checkerboard pattern are used as synonyms.
In this document, the expression a bitmap contains semi-transparent pixels usually means that the bitmap does not only consist of fully transparent pixels (ie contains at least one color pixel and optionally one or more fully transparent pixels), and is located in a window that is configured to be semi-transparent with an alpha transparency in the range of 1% to 99%, but other ways of providing a bitmap with semi-transparent pixels can also be used.
BE2018 / 5848
The present invention relates to a method for providing a graphic overlay, and to a computer program product adapted to perform such a method when executed on a computer system, and to a computer system comprising such a computer program product.
The present invention will be explained in more detail for the example of a Microsoft Windows desktop PC running a version of Microsoft Windows as the O / S (e.g., Windows XP or Windows NT or Windows 10 or Windows 12 or later Windows versions) , and running one or more applications selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a text editor; but the invention is not limited thereto, and can also be used on other computing devices, such as, for example, an Apple computer, or on portable devices such as smartphones, on which respective operating systems and applications run.
Referring to the figures,
FIG. 1 shows a schematic block diagram of a conventional computer system 100 comprising a computer device 101 connected to a keyboard 102 and a mouse 103 as input devices, and to a monitor or display or screen 104 as an output device. The computer device 101 further generally comprises storage components (e.g. in the form of flash memory or one or more hard disks) that store software (executable machine instructions), in particular, an operating system, device drivers (device drivers) and one or more applications. The computer device 101 further comprises at least one central processing unit CPU configured to execute program instructions from a multitasking operating system with a graphical user interface GUI (e.g., MICROSOFT WINDOWS 10 available from Microsoft Corporation of Redmond, Wash, US), and one or more software applications such as a text viewer or text editor (eg Microsoft Word or Notepad (Notepad) or WordPad) and / or a PDF document viewer (eg Adobe Acrobat Reader) and / or a PDF document editor (eg Nuance Power PDF), and / or an internet browser (eg Microsoft Internet Explorer or Mozilla FireFox or Google Chrome), and / or other applications.
Inputs from the keyboard 102 and the mouse device 103 are typically handled by device drivers configured to receive information via respective input ports (e.g., a serial port or parallel port or USB port). Such computer systems and application programs and device drivers are known in the art.
In the specific example of FIG. 1, the computer device 101 generates a graphic image that is displayed on the display 104 as indicated by the rectangular area 109
BE2018 / 5848 (slightly offset from the display border shown for illustrative purposes). The graphic image comprises image parts related to a so-called desktop window (in this example visible on the left-hand side of the screen 104), and related to an application window (e.g. MS Word) that shows a text document (in this example on the right-hand side of the screen 104) ). In the specific example of FIG. 1, the desktop window includes an image of a car 107 and three icons 108, each icon representing an application, but this is only an example. Also shown is a so-called mouse pointer 199 or mouse cursor that can take various forms, and which moves in accordance with movements of the mouse device 103, under control of the control system.
FIG. 2 is a schematic representation of a possible Z-order of three windows or image planes 281, 282, 283 that can be used in the computer system of FIG. 1, to obtain the combined image 109 shown on the display 104 of FIG. 1. The first window 281 with the lowest Z-order Z1 is called the desktop window, and is usually generated by the operating system. The second window 282 with a second Z-order Z2 higher than Z1 is associated with application 105, (in the example MS Word), and displays alphanumeric or textual information 106. The third window 283 with a Z-order Z3 higher than Z2 is referred to herein as the mouse bitmap 199. In the example, the mouse bitmap 199 is represented by a relatively small window 283, e.g. with a rectangular shape of about 18x22 pixels, or a square shape of 32x32 pixels, comprising a plurality of first pixels P1 showing a white arrow with a black border, surrounded or partially surrounded by a plurality of second pixels P2 that are completely transparent pixels. The desktop image 281 and the application image 282 are generally opaque images. The three image planes 281, 282, 283 are combined in known ways, resulting in the combined image shown on the display 104. This combined image can be captured, for example, using a well-known print screen function.
FIG. 3 shows a schematic block diagram of a computer system 300 that may have the same hardware configuration as the system 100 of FIG. 1. The system 300 has storage components that store executable instructions from an operating system with a graphical user interface (e.g., Windows 10) and a text editor application (e.g., MS Word), and with a specific overlay application known in the art as LineReader (which can be downloaded from http://www.iconico.com/lineReader/ at the time of writing this document). The overlay application shows a semi-transparent line 310 that follows movements of the mouse pointer 399. The line 310 can be used by a user to underline text fragments 306. In fact, LineReader also comes with a user interface, though
BE2018 / 5848 that aspect is not relevant to the present invention and is therefore not shown or further discussed.
FIG. 4 is a schematic representation of a possible Z-order of four windows or image planes 481-484 as may be used in the computer system 300 of FIG. 3, to obtain the combined image 309 shown on the display 304 of FIG. 3. As far as the inventors are aware, FIG. 4 is not publicly available in the state of the art, but is a suspected implementation.
The first window 481 on Z1 is called the desktop window. The second window 482 on Z2 is associated with application 305 (in the example MS Word) and displays alphanumeric or textual information 306. A third window 483 is associated with the LineReader application and has Z-order Z3 higher than Z2 of the text editor application 305. The mouse bitmap or mouse image plane 399 is located in window 484 with Z-order Z4 higher than Z3.
The desktop image 481 and the application image 482 are generally opaque images. The window 483 contains a small line or bar 310, and is configured in click-through mode and in semi-transparent mode with a configurable alpha-mix value that makes it possible to change the transparency level of the line or bar 310. As in FIG. 2, the mouse image plane 484 has first pixels P1 showing a white arrow with a black border, at least partially surrounded by a plurality of second pixels P2 that are fully transparent pixels. The four image planes 481-484 are combined in known ways, resulting in the combined image 309 shown on the display 304. A portion of the bar is shown enlarged to show that all the pixels P3 of the line 310 are of the same color, for example red or blue.
FIG. 5 shows a schematic block diagram of a computer system 500 similar to that of FIG. 1 when it is running an operating system O / S and two applications: a text editor and a specific overlay application known in the art as F.Lux, (which can be downloaded from https at the time of writing this document : //www.microsoft.com/en-us/store/p/flux/9n9kdphv91jt), the latter displaying a semi-transparent screen-filling overlay window, created according to the same website in 2008, almost a decade ago.
FIG. 6 is a schematic representation of a possible Z-order of four windows or image planes 681-684 that can be used in the computer system of FIG. 5, to obtain the combined image 509 shown on the display 504 of FIG. 5. As far as the inventors are aware, FIG. 5 is not publicly available in the state of the art, but is a suspected implementation.
BE2018 / 5848
The first window 681 on Z1 is called the desktop window. The second window 682 on Z2 is associated with application 505 (in the example MS Word) and displays alphanumeric or textual information 506. A third window 683 at height Z3 is associated with the F.Lux application and has Z-order Z3 higher than Z2 of the text editor application 505. The mouse bitmap or mouse image plane 599 is located in window 684 with Z-order Z4 higher than Z3.
The desktop image 681 and the application image 682 are generally opaque images. As far as known to the inventors of the present invention, the window 683 contains a monochrome bitmap with pixels P4 which all have the same color. The window 683 is configured in click-through mode (ie, it sends all mouse events to child layers, in this example: to the desktop window 681 or to the text editor window 682, depending on the position of the mouse cursor) and is configured in semi -transparent mode with a configurable alpha mix value. As in FIG. 2, the mouse image plane 684 has first pixels P1 showing a white arrow with a black border, at least partially surrounded by a plurality of second pixels P2 that are fully transparent pixels. The four image planes 681-684 are combined in known ways, resulting in the combined image 509 shown on the display 504.
The computer system of FIG. 3 (with the LineReader application) can be helpful for a user when reading textual information on the screen, but is not ideal when extracting information from tables, or when editing a worksheet.
FIG. 7 shows a schematic block diagram of a computer system 700 according to an embodiment of the present invention. The computer system comprises a computer device 701, and a keyboard 702 and a pointing device or position device 703 (e.g., a mouse device or a touch pad or a joystick) and a display 704 connected to said computer device 701.
The computer device 701 includes a storage or memory device (not shown) that contains an operating system O / S with a graphical user interface GUI (e.g., Windows XP or Windows NT or Windows 10 or Windows 12 or later versions of Windows), and at least one application selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a text editor.
The computer device 701 further comprises an overlay application showing a semi-transparent overlay window 883 that comprises at least two semi-transparent elements: i) a horizontal line 721 or beam, and ii) a vertical line 722 or beam. The horizontal line 721 preferably extends substantially over the entire width Wo of the overlay window 883, which is preferably equal to the width Wd of the pixel area or the working area of the screen. The vertical line 722 or beam preferably extends substantially over the entire height Ho of the overlay window 883, which at
BE2018 / 5848 is preferably equal to the height Hd of the pixel area or the working area of the screen, regardless of the actual size of any underlying application window 705 (e.g. MS Excel). The overlay application, even though it is configured in click-through mode, ensures that the horizontal line
721 and the vertical line 722 move in accordance with movements of the input device 703, or more specifically, that the horizontal line 721 moves in accordance with the Y coordinate of the mouse pointer 799 and that the vertical line 722 in accordance with the X coordinate of the mouse pointer 799. Preferably, the horizontal line 721 and the vertical line 722 are positioned such that the mouse pointer 799 is at least partially on both the horizontal line 721 and the vertical line 722, for example such that the pointed end of the mouse pointer 799 is in or near the center of the rectangular intersection of the horizontal line 721 and the vertical line
722. Expressed in simple terms, this embodiment provides an overlay application with a semi-transparent cross that moves with the mouse pointer 799.
FIG. 8 is a schematic representation of a possible Z order of four windows or image planes 881-884 as may be used in the computer system 700 of FIG. 7, to obtain the combined image 709 shown on the display 704 of FIG. 7.
The first window 881 at height Z1 is called the desktop window. The second window 882 on Z2, larger than Z1, is associated with application 705 (in the example MS Excel), which usually presents alphanumeric and / or textual information organized in rows and columns. A third window 883 is associated with an overlay application according to an embodiment of the present invention, and has a Z-order Z3 higher than Z2 of the spreadsheet application 705. The mouse bitmap or mouse image plane 799 provided by the O / S with graphic user interface GUI is located in window 884 on Z4, higher than Z3.
The desktop image 881 and the application image 882 are generally opaque images. The window 883 provides a substantially screen-filling overlay, configured in click-through mode and also configured in semi-transparent mode with a configurable alpha blend value α (also called alpha transparency) in the range of 5% to 95%, or from 10% to 90% or from 20% to 80%, or from 30% to 70%, where α = 0% means that the line is completely transparent, and α = 100% means that the line is completely opaque. The value of α should not be too low, because otherwise the cross is almost invisible and hard to find. The value of α should not be too high, because otherwise the cross is almost opaque, and the information under the cross is obscured. The person skilled in the art and / or the user can find suitable values.
Preferably, the overlay application also has another window (not shown in FIG. 7 and FIG. 8, but see, for example, FIG. 35 for an example) as a user interface. This user interface window is then preferably higher than Z3 of the click-through window 883. Preferably, this user interface window makes it possible to define various features of the
BE2018 / 5848 cross, such as for example: color, width of the lines that form the cross and transparency level.
Referring back to FIG. 8, is a majority portion of the semi-transparent window
883 occupied by fully transparent pixels P5. A minority portion of the semi-transparent window 883 is occupied by the movable horizontal line 721 and the movable vertical line 722. The lines 721, 722 can be implemented as purely rectangular areas, with or without an edge (e.g. a black edge), and with or without rounded edges, and with or without other features. The lines 721, 722 may include monochrome pixels, or may include a color gradient. The color of both lines is preferably the same, and adjustable or selectable. In preferred embodiments, the height hh of the horizontal line 721, and the width ww of the vertical line 722 are adjustable. The values of hh and ww are preferably the same (which simplifies user settings), but that is not absolutely required. As in FIG. 2, the mouse image plane 884 has first pixels P1 showing a white arrow with a black border, at least partially surrounded by a plurality of second pixels P2 that are fully transparent pixels. The accumulation of four image planes 881 to 884 is combined in known ways, resulting in the combined image 709 shown on the display 704.
This overlay application is extremely suitable for finding a correct cell, because the horizontal line 721 and the vertical line 722 extend above the respective row and column headings. This allows a user to immediately see whether or not the mouse pointer is positioned on or above the correct cell without having to click on any column headings and / or row headings. This overlay application can be seen as an extremely useful add-on for spreadsheet applications, but can also be used when extracting information from tables presented on web pages. The tool is especially useful on high resolution screens (with a resolution of at least 1920 x 1080 pixels).
The tool can drastically reduce the cognitive burden of a user trying to find a particular cell located at the intersection of a certain row (with a certain row header) and a specific column (with a specific column header). Without the overlay tool with the semi-transparent cross, many users first tentatively click on a cell in the vicinity of where they think the target cell is likely to be, and then verify that the correct cell was selected by starting from the selected cell (which is after the is highlighted) and then move his or her eyes horizontally to the row headings trying to stay in the same row as the selected cell, and if the row is not correct, click on the cell higher or lower than the selected cell , and repeat the process, and then perform a similar check to verify that the selected cell is in the correct column by starting again from the selected cell, and moving his or her eyes up, to the column. headings,
BE2018 / 5848 and if it appears that the column is not correct, select a cell to the left or right of the selected cell, and optionally double-check the finally selected cell. This takes time and is very error prone. It is of course possible to use other ways, for example by panning, or by selecting a specific column by clicking on the column heading, or by using freeze panes (a property in Excel around a certain number of rows and / or block columns), but this also takes time, and introduces the risk of inadvertently changing the height of a row, or the width of a column, etc. By contrast, with the overlay tool presented in FIG. 7, the user can simply move the cross to be positioned above the correct row header and column header, and then click to directly select the correct cell. In addition, the tool makes it possible to quickly double-check the selected position by moving his or her eyes immediately to the row and column headings, without having to try to stay in the same row or column.
Although no objective evaluation tests have been performed, it is expected that the overlay tool can help improve performance and / or concentration for people who frequently use spreadsheets, and / or that the risk of extracting or inserting information from / into an incorrect cell is drastically is reduced.
The inventors of the present invention are of the opinion that it is not obvious or trivial to arrive at this solution if one takes proper account of the fact that more than 100 million people use spreadsheets every day for almost two decades. If the solution proposed herein were obvious, it would have been widely available for many years.
Instead of providing a window 883 with fully transparent pixels P5 and two movable objects in the form of a horizontal line 721 and a vertical line 722, the same visual effect can also be implemented in other ways, for example by providing a window surface 883 with a screen-filling bitmap with pixels with the color of the cross (e.g. red), and by placing on top of this screen-filling bitmap four other bitmaps, each extending from one of the four corners and having fully transparent pixels, and by adjusting the width and height of each of these four rectangles in accordance with the cursor position 799, or in accordance with movements of the pointing device 703, so as to emulate the cross as the area not overlaid by the four rectangular bitmaps; or in other suitable ways.
The computer system of FIG. 3 (with the LineReader application) can help with reading textual information, but does not help to reduce eye fatigue caused by, for example, a clear white background. The computer system of FIG. 5 (with the F.Lux overlay application) can help to reduce eye fatigue, but does not provide a line
BE2018 / 5848 or bar to underline text. It would be nice to have a single overlay application that offers both properties at the same time.
As far as known to the inventors, there is currently no computer application tool available on the market that provides an overlay that simultaneously (1) allows to reduce the brightness of a clear (e.g. white) background, and ( 2) showing a semi-transparent line or a semi-transparent cross that follows mouse movements. Initial experiments have shown that a mere combination of the overlay surfaces shown in FIG. 4 and in FIG. 6 does not produce acceptable results, as can be understood from FIG. 13 (d) to FIG. 13 (e), due to conflicting requirements. Indeed, for the text information of the underlying document to be easily legible (e.g., by maintaining sufficient contrast), the transparency level of the third window 883 must be sufficiently high (e.g., T> = 80% or α <= 20% in FIG. 13 (d) to FIG. 13 (e)). At the same time, for the line to be easily recognizable and the text below the line to be readable, the transparency level of the third window 883 must be sufficiently low (e.g., T in the range of 30% to 70%). The exact figures can be discussed, but the expert reader will agree (after implementing the method described above on a computer) that no acceptable or attractive combination can be found, or at least that such a solution is a heavy compromise.
There is a need for a method to simultaneously reduce the brightness of a clear background and to show a semi-transparent line or a semi-transparent cross that follows mouse movements. The combination should be realized in such a way that (1) text information retains good readability, both text information that is overlaid by the line, as well as text information that is not overlaid by the line, and that (2) the semi-transparent line simply from the background can be distinguished, but which is still sufficiently transparent to maintain the readability of the text below the line.
The present invention meets this need by providing both a high level of transparency (high T, low α) for the (e.g., screen-filling) bitmap, and at the same time providing a low level of transparency (low T, high α) for the line , by perforating the bitmap, for example as illustrated in FIG. 10 and FIG. 29. A perforated bitmap is understood to mean a bitmap with a first plurality of pixels P6 that are completely transparent (e.g., have a pseudo-color value), and with a second plurality of pixels P7 that have one or more true color values. The pixels P7 will be alpha mixed with underlying layers. As can be understood from FIG. 13 (f) to FIG. 13 (i) this combination provides functional results for almost all transparency levels from 5% to 95% and almost every color or intensity of the pixels P7, but a preferred result is obtained for transparency levels of about 30% to about 70%. Depending on the intended use, a user can configure the transparency level to a more preferred level.
BE2018 / 5848
The combination of the two semi-transparent line properties to underline textual information to reduce cognitive burden and the semi-transparent bitmap to change the color temperature to reduce eye fatigue when combined together create a technical challenge related to how a semi-transparent combining a transparent bitmap and a semi-transparent line such that the transparency level of the bitmap is higher than the transparency level of the line. This presents a technical problem related to image overlay, image blending, semi-transparency, and alpha mixing, which is a problem rooted in computer technology, and the solution proposed by the present invention (using a perforated bitmap) is also rooted in computer technology. This technology can be used to easily read textual information on a screen with less eye strain.
It is further noted that it is not the overlay application that generates or displays textual or alphanumeric information, but the one or more underlying application (s), eg a text editor, a spreadsheet, a web browser etc. The method of overlaying therefore presents no information.
Even though the overlay application can be run on a standard computer with a standard O / S (e.g., Windows 10 from Microsoft Corporation), the solution proposed herein provides a technical effect related to two different levels of transparency in a single semi-transparent window, which effect is anything but standard, and goes far beyond mere electrons flowing through a transistor. Furthermore, at least a part of the functionality can be implemented in software on the central processing unit or units of the computer system, or at least a part of the functionality can be implemented in a graphic processing unit, if present in the computing device, in particular the alpha mixing operation, and / or time multiplexing as will be further described when discussing FIG. 28.
FIG. 9 shows a schematic block diagram of a computer system 900 according to an embodiment of the present invention. The computer system 900 includes a computer device 901, and a keyboard 902 and a pointing device or position device 903 (e.g., a mouse or a touch pad or a joystick or a stylus) and a display 904 connected to said computer device 901.
The computer device 901 includes a storage device (not shown) that includes an operating system O / S with a graphical user interface GUI (e.g., Windows XP or Windows NT or Windows 10), and at least one application selected from the group consisting of: a web browser application, a spreadsheet application, a PDF document viewer, a PDF document editor, a text viewer and a text editor. Instead of being stored in the
BE2018 / 5848 computer device itself, the O / S and / or the at least one application may be downloadable from a server, e.g. via a network interface. (this also applies to other embodiments of the present invention).
The computer device 901 further comprises an overlay application showing a semi-transparent overlay window 1083 comprising at least two semi-transparent elements: i) a so-called perforated bitmap 930 occupying a majority portion of the overlay window 1083, and ii) a block or a line 910 which occupies only a minority portion of the overlay window 1083. The line 910 is movable in accordance with movements of the input device 903. Instead of being stored in the computer device itself, the overlay application can be downloadable from a server, e.g. via a network interface.
The overlay window 1083 is configured in click-through mode and in semi-transparent mode with a predefined or adjustable alpha mix value α.
The bitmap 930 contains a first plurality of pixels P6 that are completely transparent, and a second plurality of pixels P7 with a color value that will be alpha mixed with the graphic image formed by the accumulation of underlying windows (in the example windows 1081 and 1082). In preferred embodiments, the first plurality of pixels P6 are alternately positioned (interleaved) with the second plurality of pixels P7, in a checkerboard pattern, as shown in FIG. 10 (see also FIG. 29). So preferably 50% of the pixels are completely transparent, while the others are 50% semi-transparent. The effect of using this perforated bitmap is that the spatially average color of the bitmap has a seemingly higher level of transparency than would be the case without the perforation. In this way, the apparent transparency of the bitmap 930 is made higher than that of the line 910, despite the fact that both are associated with the same window 1083 that has only a single alpha merge value α.
As was the case for the overlay application of FIG. 7, the overlay application of FIG. 9 usually with a user interface, which is shown in a second, opaque window (not shown in FIG. 10 but see, for example, FIG. 35). In the embodiment of FIG. 10, the user interface will typically allow a user to change the size (height and / or width) of the line 910, the transparency level α, the color of the pixels P3 of the line 910, the color of the second plurality of pixels P7, can adjust.
The overlay application, even though one of its windows 1083 is configured in click-through mode, causes the line 910 to move in accordance with movements of the input device 903. This can be realized, for example, by repeatedly positioning the native mouse pointer 999 on to ask the operating system (eg based on a timer, or based on polling).
Preferably, the line 910 is positioned such that the mouse pointer 999 is near the center of the line, but this is not absolutely required and the mouse pointer may be somewhat
BE2018 / 5848 are shifted relative to this position, eg above the line or below the line. Alternatively, the mouse pointer may also be positioned at or near the left end of the line, or at or near the right end of the line. In any case, the overlay application provides a semi-transparent line 910 that moves with the mouse pointer 999, both of which are visible.
FIG. 10 is a schematic representation of a possible Z-order of four windows or image planes 1081-1084 as may be used in the computer system 900 of FIG. 9, to obtain the combined image 909 shown on the display 904 of FIG. 9.
The first window 1081 on Z1 is called the desktop window. The second window 1082 on Z2 is associated with application 905 (in the example a text editor application, e.g. MS Word), and displays alphanumeric or textual information 906. A third window 1083 is associated with an overlay application according to an embodiment of the present invention and has a Z-order Z3 higher than Z2 of the text editor application 905. The mouse bitmap 999 or the mouse image plane is located in window 1084 at Z4, higher than Z3 .
The desktop image in the first window 1081 and the application image in the second window
1082 are usually opaque images. The window 1083 provides a substantially screen-filling overlay configured in click-through mode and also configured in semi-transparent mode with a configurable alpha blend value α. A minority portion of the overlay window 1083 is occupied by pixels P3 with the line color (e.g. red), a majority portion is occupied by the perforated bitmap, in the example a bitmap with a first plurality of pixels P6 that are fully transparent pixels, and a second plurality pixels P7 with a predefined constant color. It is noted that such a perforated bitmap can be efficiently implemented by, for example, first generating a small bitmap of for example 64 x 64 pixels (or 100 x 100 pixels or any other suitable size), and then copying or tiling this small bitmap ( tiling) in the substantially screen-filling bitmap 1083, but other ways can also be used. In an alternative embodiment, this functionality can be implemented as a functionality in GPU chips, thus requiring only minimal load on the central CPU of the computing device.
The line 910 can be implemented as a purely rectangular area, with or without a border (e.g., a black border), and with or without rounded edges or borders, and with or without other features (e.g., with a color gradient).
The overlay application of the computer system 900 is extremely suitable for reading large amounts of text on the screen. In particular, the overlay application can be used to change the color (or color temperature) of the underlying graphic image, and / or its brightness, which can help reduce eye fatigue, and provides them with a
BE2018 / 5848 movable line 910 that a user can use to highlight or underline text displayed in one or more underlying windows.
In certain embodiments, the color temperature can be changed to reduce the amount of blue light. In other embodiments, the color temperature can be changed to increase the amount of blue light.
Although the same alpha mixing value α is used for the line 910 as for the bitmap 930, the bitmap 930 is more transparent than the line, viewed from a distance where the eye does not distinguish individual pixels, but averages pixel information spatially.
For the sake of completeness, it is noted that other perforation patterns were tested (for example, where 4 of the 16 pixels are fully transparent, or 12 of the 16 pixels are fully transparent), but the checkerboard pattern with 50% fully transparent pixels may be preferred as it appears to be the best results to provide in terms of least noticeable artifacts.
FIG. 11 shows a variant of the computer system of FIG. 9, wherein the semi-transparent line 1120 of the overlay application extends over substantially the full width of the screen (e.g., the full width of the pixel area or the full width of the working area of the screen). This implies that the line 1120 only has to follow vertical movements of the input device 1103, not horizontal movements.
In the example shown, two applications show textual information, for example a PDF document viewer on the left side of the screen, and a text editor window on the right side of the screen. This can be used, for example, when comparing two documents side by side, for example when proofreading a translation, where the source text is in the PDF document and the target text is in the text document. In the example, the window presents
1282 at height Z2 information associated with the text editor application, and presents the window
1283 up to date Z3 information associated with the PDF document viewer.
FIG. 12 is a schematic representation of a possible Z-order of four windows or image planes 1281 - 1284 as may be used in the computer system 1100 of FIG. 11, to obtain the combined image 1109 shown on the display 1104 of FIG. 11. The area for the mouse cursor is not shown, and the area for the user interface window of the overlay application (if present) is not shown, so as not to overload the drawings. Preferably, however, the color of the pixels P3 and of the pixels P7, and the transparency level α, and the height of the line 1120 is adjustable via a user interface, for example a user interface
BE2018 / 5848 similar to the exemplary user interface of FIG. 35 without a slider for selecting the width of the line.
FIG. 13 shows a plurality of exemplary screenshots (screen recordings). Each row shows a series of five images corresponding to an alpha mix value α of 100%, 80%, 60%, 40% and 20% respectively, corresponding to a transparency value T of the overlay window of 20%, 40%, 60%, respectively 80% and 100%, where α = 100% or T = 0% means that the overlay image is completely opaque (except for fully transparent pixels), and α = 0% or T = 100% means that the overlay image is completely transparent, and values from 1% to 99% or from 2% to 98% or from 5% to 95% means that the overlay image is semi-transparent, meaning that the pixels of the overlay image are proportionally mixed with the underlying image (except for full transparent pixels).
Each of the images shown in FIG. 13 (a) to FIG. 13 (e) contains a text fragment overlayed with two red lines with color (R, G, B) = (255, 0, 0). Two lines are used for illustrative purposes, to illustrate both highlighting and underlining of text in the same drawing.
In FIG. 13 (a) each text fragment is overlayed by two red lines with (R, G, B) = (255, 0, 0). The rest of the overlay window is completely transparent. This series of images shows results that are similar to what would be obtained by the overlay application of FIG. 3. As can be understood to provide a line that is well suited to highlighting and / or underlining underlying textual information without obscuring it, the level of transparency should preferably be in the range of about 30% to about 75%. For lower values of T, the text becomes unreadable. The line is hardly visible for higher values of T. (It is noted that a proper evaluation should be performed on a screen rather than on paper, but the paper version allows understanding of the principles of the present invention).
In FIG. 13 (b), each text fragment is covered (overlayed) with a semi-transparent bitmap containing only light gray pixels with (R, G, B) = (192, 192, 192), and in FIG. 13 (c) each text fragment is covered (overlayed) with a semi-transparent bitmap that only contains dark gray pixels with (R, G, B) = (64, 64, 64). This series of images shows results similar to what would be obtained by the overlay application of FIG. 5. In order to maintain good readability of the underlying text, the transparency level T should preferably be at least about 85% (or α <= 15%).
In FIG. 13 (d), each text fragment is overlayed (covered) by a mere combination of the two red lines as in FIG. 13 (a) and through a light gray bitmap as in FIG. 13 (b), and in FIG. 13 (e)
BE2018 / 5848, each text fragment is overlayed (covered) by a mere combination of the two red lines as in FIG. 13 (a) and through a dark gray bitmap as in FIG. 13 (c).
As described above, when the transparency level T is very low (e.g. <30%), the line is not sufficiently transparent and the text underneath is blurred; if the transparency level T is mediocre (e.g. from about 30% to about 70% or about 75%), then the line is well visible and sufficiently transparent, but the text is not easily readable; and if the transparency level T is too high (for example, T> 85%), then the text is easy to read, but the line is almost invisible. None of these combinations therefore provide a good solution for both the line and the bitmap.
In FIG. 13 (f), each text fragment is overlayed by two red lines as in FIG. 13 (a) and by a so-called perforated bitmap as described above, wherein 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are light gray pixels P7, with (R, G, B) = (192, 192, 192). As can be understood from the drawings (although of course this should be evaluated on a real screen), very acceptable results for both the line and the bitmap are obtained for transparency levels T of about 30% to about 75%.
In FIG. 13 (g), each text fragment is covered (overlayed) by two red lines as in FIG. 13 (a) and by a so-called perforated bitmap as described above, wherein 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are dark gray pixels P7, with (R, G, B) = (64, 64, 64). Again, very acceptable results for both the line and the bitmap are obtained for transparency levels T of about 30% to about 75%.
In FIG. 13 (h), each text fragment is overlayed by two red lines as in FIG. 13 (a) and by a so-called perforated bitmap as described above, wherein 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are black pixels P7, with (R, G, B) = (0, 0, 0). As can be understood, the text overlayed by the perforated bitmap remains surprisingly very well legible, despite the fact that 50% of the bitmap pixels are pure black, thanks to the perforation.
In FIG. 13 (i), each text fragment is overlayed by two red lines as in FIG. 13 (a) and through a so-called perforated bitmap as described above, wherein 50% of the pixels are fully transparent pixels P6 arranged in a checkerboard pattern, and the other 50% of the pixels are white pixels P7, with (R, G, B) = (255, 255, 255). As can be understood, the text remains that
BE2018 / 5848 is overlayed by the perforated bitmap, very surprisingly very easy to read, despite the fact that 50% of the bitmap pixels are pure white thanks to the perforation.
As can be understood from all screenshots in FIG. 13 (f) to FIG. 13 (i), it can be understood that the color of the non-fully transparent pixels P7 of the perforated bitmap can be varied over a wide range (from pure black to pure white), and that it is not critical to the readability of the underlying textual information.
Although the inventors do not wish to be bound by any theory, FIG. 14 and FIG. 15 help to better understand the difference between mixing with a monochrome bitmap versus mixing with a perforated bitmap that contains 50% fully transparent pixels. The intensity (I) is shown on the vertical axis, the horizontal axis is transparency T or alpha transparency α, where α + T = 100%.
FIG. 14 (a) illustrates what happens to a graphic image containing text information with Original Text pixels OT (black) with color (0, 0, 0) and Original Background pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a monochrome bitmap (as in FIG. 6) containing 100% black pixels with color (0, 0, 0), for different values of the transparency T or alpha mix value α.
In general, the result of overlaying or mixing two bitmaps is a color with the following color components:
(Rm, Gm, Bm) = (R1, G1, B1) x (1-T) + (R2, G2, B2) xT [1] where Rm, Gm, Bm are the color components of pixels of the resulting image (the mixed image), T is the transparency level, (R1, G1, B1) is the color of a pixel of the first (overlay) bitmap, and (R2, G2, B2) is the color of a pixel of the second (underlying) bitmap.
When mixing with a black monochrome bitmap, then (R1, G1, B1) = (0, 0, 0).
If T = 100%, then originally white background pixels OB (255, 255, 255) and originally black text pixels OT (0, 0, 0) remain unchanged after mixing.
If T = 50%, originally white background pixels OB (255, 255, 255) gray pixels MB (127, 127, 127) are omitted from rounding errors, and originally black text pixels remain black MT (0, 0, 0).
If T = 0%, then originally white background pixels OB (255, 255, 255) and originally black text pixels OT (0, 0, 0) both turn black (0, 0, 0).
This is more or less visualized by the series of FIG. 13 (c), except that in FIG. 13 (c) the monochrome bitmap has pixels P4 with color (64, 64, 64) instead of (0, 0, 0).
BE2018 / 5848
FIG. 14 (b) illustrates what happens to a graphic image containing text information with Original Text pixels OT (black) with color (0, 0, 0) and Original Background pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a monochrome bitmap (as in FIG. 6) containing 100% white pixels with color (255, 255, 255), for different values of the transparency T or alpha mix value α.
The same mixing formula [1] applies, but now (R1, G1, B1) = (255, 255, 255).
If T = 100%, then white pixels (255, 255, 255) and black pixels (0, 0, 0) remain unchanged after mixing.
If T = 50%, then white pixels (255, 255, 255) remain white (255, 255, 255) and black pixels (0,
0, 0) turns gray pixels (127, 127, 127) apart from rounding errors.
If T = 0%, then white pixels (255, 255, 255) and black pixels (0, 0, 0) become both (255, 255, 255).
This is more or less visualized by the series of FIG. 13 (b), except that in FIG. 13 (b) the monochrome bitmap has pixels P4 with color (192, 192, 192) instead of (255, 255, 255).
FIG. 15 (a) illustrates what happens to a graphic image containing text information with Original Text pixels OT (black) with color (0, 0, 0) and Original Background pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a perforated bitmap (as in FIG. 10 or FIG. 12) containing 50% fully transparent pixels P6, and 50% black pixels P7 with color (0, 0, 0), for different values of the transparency T (or alpha mix value α).
In general, the result of overlaying or mixing with a perforated bitmap with 50% fully transparent pixels P6:
(Rm, Gm, Bm) = (R1, G1, B1) x (1-T) + (R2, G2, B2) xT [2] if the overlay pixel is not completely transparent, as is the case for the pixels P7, (Rm, Gm, Bm) = (R2, G2, B2) [3] if the overlay pixel is completely transparent, as is the case for the pixels P6, so on average (if the number of pixels P6, P7 is each 50%):
(Rm, Gm, Bm) = 50% * [(R1, G1, B1) x (1-T) + (R2, G2, B2) xT] + 50% * (R2, G2, B2) [4] where Rm, Gm, Bm are the color components of pixels of the resulting image (the mixed image), T is the transparency level, (R1, G1, B1) is the color of a pixel of the first (overlay) bitmap, and (R2, G2, B2) is the color of a pixel of the second (underlying) bitmap.
Specifically, when blending with a perforated black bitmap with 50% fully transparent pixels P6 and 50% black pixels P7, then (R1, G1, B1) = (0, 0, 0).
If T = 100%, then white pixels (255, 255, 255) and black pixels (0, 0, 0) remain unchanged.
BE2018 / 5848
If T = 50%, then white pixels (255, 255, 255) average become gray pixels (191,191,191), and black pixels remain black (0, 0, 0).
If T = 0%, then white pixels (255, 255, 255) on average become gray pixels (127,127,127), and black pixels (0, 0, 0) remain black (0, 0, 0).
This is more or less visualized by the series of FIG. 13 (h), if the two lines are omitted.
FIG. 15 (b) illustrates what happens to a graphic image containing text information with Original Text pixels OT (black) with color (0, 0, 0) and Original Background pixels OB (white) with color (255, 255, 255) when this image is overlayed (mixed) with a perforated bitmap (as in FIG. 10) containing 50% fully transparent pixels P6, and 50% white pixels with color (255, 255, 255), for different values of the transparency T or alpha mixed value α .
The same formulas [2] to [4] apply.
When mixing with a perforated white bitmap with 50% fully transparent pixels P6 and 50% white pixels P7, (R1, G1, B1) = (255, 255, 255).
If T = 100%, then white pixels (255, 255, 255) and black pixels (0, 0, 0) remain unchanged.
If T = 50%, then white pixels (255, 255, 255) remain white, and black pixels become average (64, 64, 64).
If T = 0%, then white pixels (255, 255, 255) remain white, and black pixels on average become gray pixels (127,127,127).
This is more or less visualized by the series of FIG. 13 (i), if the two lines are omitted.
Thus FIG. 14 can help to understand what happens to black text on a white background when it is overlayed by a monochrome (non-perforated) bitmap, as is the case for the movable object, and FIG. 15 can help to understand what happens to black text on a white background when it is overlayed by a perforated bitmap.
The following section explains what happens to black text on a white background when it is overlayed by a red line. The same formula [1] applies, but now it holds that (R1, G1, B1) is (255.0.0).
FIRST EXAMPLE:
As a first example, the colors of the image fragment of FIG. 13 (e) for T = 80% calculated.
BE2018 / 5848
The original image includes: black text (0, 0, 0) on a white background (255, 255, 255).
The overlay window includes a (non-perforated) monochrome bitmap with black pixels (0, 0, 0) and a line with red pixels (255.0.0), and mixing is done with T set to 80%. Using formula [1], it can be calculated that:
- white background pixels overlayed by the black overlay bitmap: (204, 204, 204),
- black text pixels overlayed by the black overlay bitmap: (0, 0, 0),
- white background pixels overlayed by the red line are: 80% * (255, 255, 255) +20% * (255, 0, 0) = (255, 204, 204), which is very difficult to distinguish from (204, 204, 204),
-the black text pixels overlayed by the red line are: 80% * (0, 0, 0) +20% * (255, 0, 0) = (51, 0, 0).
SECOND EXAMPLE:
As a second example, the colors of the image fragment of FIG. 13 (h) for
T = 60% calculated.
The original image includes: black text (0, 0, 0) on a white background (255, 255, 255).
The overlay includes a perforated bitmap with 50% fully transparent pixels P6 and 50% black pixels P7 (0, 0, 0), and a line with red pixels (255, 0, 0), and mixing is done with T set to T
60%. Using formulas [1] to [5], it can be calculated that:
white background pixels overlayed by the perforated bitmap on average (204, 204,
204), namely (255, 255, 255) for pixels are overlayed by fully transparent pixels P6, and (153, 153, 153) for the pixels are overlaid by a black pixel P7;
- black text pixels overlayed by the black overlay bitmap: (0, 0, 0),
- white background pixels overlayed by the red line are: 60% * (255, 255, 255) +40% * (255, 0, 0) = (255, 153, 153), which can easily be distinguished from (204, 204) ,
204)
-the black text pixels overlayed by the red line are: 60% * (0, 0, 0) +40% * (255, 0, 0) = (102, 0, 0).
Comparison of the first and second example (or the sample of FIG. 13 (e) for T = 80% and the sample of FIG. 13 (h) for T = 60%) teaches that:
- in both cases, black text on a white background (0, 0, 0) becomes an average background color of (204, 204, 204);
- in the case of a non-perforated bitmap, text is overlayed by the red line (51, 0, 0) on (255, 204, 204), which text is legible, but the line is very difficult to distinguish from the background from (204, 204, 204), and therefore not very helpful to underline,
BE2018 / 5848 while in the case of the perforated bitmap text is overlayed by the red line (102, 0, 0) at (255, 151, 153), which text is still very legible, but moreover the line can easily distinguish from the average background of (204, 204, 204).
Tests have shown that (for the settings of FIG. 13 (h), which means: a perforated bitmap with 50% fully transparent pixels and 50% pure black pixels arranged in a checkerboard pattern), very good results are also obtained for many other line colors in addition to pure red, for example a black line with color (0, 0, 0), a blue line with color (0, 0, 255), a dark red line with color (128, 0, 0), a dark blue line with color (0, 0, 128), a dark green line with color (0, 50, 0), a dark teal line with color (0, 51, 102), a green line with color (0, 128, 0 ), etc, in fact for most relatively dark lines, but also for most relatively bright lines, eg a yellow line with color (255, 255, 0), a clear green line with color (0, 255, 0) , a turquoise line with color (0, 255, 255), a light gray line of (192, 192, 192) etc. Preferably, the line color can be selected by the user via a user interface which enables the user to select from a variety of suitable colors that produce good results.
While the examples in FIG. 13 are provided for an overlay bitmap (perforated or non-perforated) with black, dark gray, light gray or white pixels, the invention is not limited thereto, and the invention also works when these pixels are not gray-scale pixels, but are, for example, bluish pixels or yellowish pixels ( e.g. with color (81, 69, 44) or (83, 69, 40) or (86, 69, 35) etc) or reddish pixels, or basically any other color. In this way the originally white background color can be transformed into a bluish or yellowish or reddish ... background, which can be more pleasant to read. In particular, yellowish pixels tend to provide a paper-like appearance, especially in combination with a texture bitmap (see below), and bluish pixels tend to provide an airy or day-like appearance, etc., for both the perforated bitmap and the non-perforated bitmap. perforated bitmap. This technique can also be used, for example, to reduce the blue light content emitted by a display, which according to some sources can influence the sleep of the user.
FIG. 16 to FIG. 18 show three examples of a slightly larger text fragment, allowing a better understanding of how a reader would view the textual information on a screen.
FIG. 16 is a screenshot (screen capture) of a text fragment extracted from US6333753B1, presented by a PDF viewer called Nuance PDF Reader, where the document is scaled to 200% in the PDF viewer, displayed on a display with a native resolution of 3840 x 2160 pixels, configured with a resolution of 2560 x 1440 pixels, and where the text and applications are scaled to 100% in Windows screen settings. FIG. 17 to FIG. Is 18
BE2018 / 5848 a screenshot of the same text fragment, but overlayed by an overlay application according to the present invention.
FIG. 16 shows an exemplary text fragment that contains black text on a white background. In fact, many applications do not display textual information as pure black pixels, but also use gray-scale pixels, especially near the edges of alphanumeric characters. This aspect is known, but is not particularly relevant to the present invention.
FIG. 17 shows the text fragment of FIG. 16, overlayed by an overlay window containing a blue line segment containing pixels with the color (R, G, B) = (0, 0, 255), and with the transparency level T of the overlay window set to 50%. The transparency level T only affects the transparency of the line, not the fully transparent overlay pixels. Such an image can probably also be obtained by the LineReader application, as discussed in FIG. 3 and FIG. 4.
FIG. 18 shows the text fragment of FIG. 16, overlayed by a method according to the present invention, as described in FIG. 9 and FIG. 10. The alpha mixing value is 50%, a minority portion of the overlay image is occupied by a blue line segment with color value before mixing (R, G, B) = (0, 0, 255), a majority portion of the overlay window is occupied by a perforated bitmap, this bitmap having 50% pure transparent pixels P6 and 50% gray pixels P7 with color value before mixing (R, G, B) = (128, 128, 128) arranged in a checkerboard pattern. As can be understood from FIG. 18, the resulting textual information is easy to read, the line can help the user to stay focused, and the line can be easily distinguished from the background.
However, the inventors went one step further, and also experimented with overlaying textures (perforated or non-perforated). Tests have shown that overlaying with certain textures can transform a pure white background (which appears shiny) into a matte background, which can be much more pleasant to read. In one experiment, an image was made of a misty sky, a sub-image was extracted, a horizontal and vertical gradient was removed, the average color was adjusted, the contrast between the brightest pixels and the darkest pixels was adjusted, and the sub-image was adjusted as a semi-transparent overlay tiled over the screen, yielding surprisingly good results, in particular because it apparently removes the gleaming white background of most text documents and replaces it with a matte background, while the text remains surprisingly easy to read.
If no semi-transparent line or cross is added in the overlay (eg for underlining text to be read, or for indicating the position of a cell in a table or
BE2018 / 5848 spreadsheet), good results can be obtained by choosing a relatively high transparency value (eg T in the range of approximately 80% to 99% for an unperforated bitmap, or T in the range of approximately 60% to 99% for a perforated bitmap).
If a line or cross is added in the overlay window, it may be better to perforate the bitmap as described above. Surprisingly, the texture pattern can even be made very visible with a perforation of 50%, and aesthetically attractive results can be obtained. However, tests have shown that a single texture bitmap appears to be unsuitable for all levels of transparency unless some modification of the pixel values is made. In both cases (perforated or not), the contrast of the texture bitmap must be adjusted or adjusted as a function of the transparency level T or alpha-transparency level α of the overlay window.
In preferred embodiments, the transparency level T of the overlay window is selectable or adjustable by the user, and the contrast level of the texture bitmap is automatically adjusted as a first or second order polynomial of the transparency level T. The value of the contrast itself is not critical (this may typically range from about 0.5 times an optimum value to about 2.0 times the optimum value, but it is important that this value is not fixed but can be adjusted when the transparency of the overlay window is varied. otherwise the texture may be too pronounced and disturbing to the reader, or almost invisible and look almost the same as a monochrome color overlay. Of course it is also possible to use multiple texture bitmaps in the overlay application, but that consumes more memory space, and can be avoided.
The inventors have found that good results, e.g. near-optimal results, are obtained when the contrast is adjusted so that the texture or pattern is barely visible, or slightly above barely visible. If the contrast level below this ideal value range is chosen, then a white background is converted into a shiny (eg glossy plastic-like) background without the texture or pattern being noticeable. If the contrast level above this ideal value range is chosen, the pattern or texture becomes dominant and can distract the user. However, if the contrast level is chosen within the ideal value range, the pattern or texture will cause a slight variation of the background color, which is perceived or experienced as a texture or a pattern or a matte color, and this even appears to be the sharpness of the text characters increase below that. The ideal value range can be different for each pattern bitmap, and is preferably also tunable by the user, in addition to the automatic adjustment related to the transparency value T.
FIG. 19 shows a schematic block diagram of a computer system 1900 according to an embodiment of the present invention, which can be seen as a variant of the
BE2018 / 5848 computer system of FIG. 10 and the computer system of FIG. 12. Most of what has been described above also applies here, except that the overlay image of the computer system 1900 of FIG. 19 comprises at least two semi-transparent elements: i) a (non-perforated) line 1910, and ii) a perforated texture bitmap 1930. The line 1910 is movable in accordance with movements of the input device 1903.
The perforated texture bitmap can be derived from a predefined texture bitmap (see, for example, FIG. 27) by assigning a pseudo-color value corresponding to a predefined full transparency value to half of the pixel locations, that is, the pixels P6, and by optionally changing the average color of the other pixels P8, optionally adjusting the contrast of the pixels P8, and optionally tiling said bitmap to fill the area of the overlay window 2083.
FIG. 20 is a schematic representation of a Z order of four windows or image planes 2081-2084 that can be used in the computer system of FIG. 19.
FIG. 21 shows a schematic block diagram of another computer system 2100 according to an embodiment of the present invention, which computer system comprises an overlay application for providing an overlay window comprising at least one semi-transparent object in the form of a so-called perforated texture bitmap with a first plurality of fully transparent pixels P6 and a second plurality of opaque or semi-transparent pixels P8. This embodiment is a variant of the embodiment of FIG. 21, the line 1910 being omitted.
FIG. 22 is a schematic representation of a Z order of four windows or image planes 2281-2284 that can be used in the computer system of FIG. 21. As described above, the overlay application may also have a second, opaque user interface window (not shown), where a user can select an average color of the pixels P8, and / or reorder the alpha transparency value α, and / or manually. can adjust a contrast of the texture bitmap, and / or select a texture bitmap from a list of texture bitmaps.
FIG. 23 shows a schematic block diagram of another computer system 2300 according to an embodiment of the present invention, which computer system comprises an overlay application for providing an overlay window comprising at least one semi-transparent object in the form of a so-called non-perforated texture bitmap consisting of a plurality of semi-transparent pixels P8. This embodiment is a variant of the embodiment of FIG. 23, wherein the perforated pixels P6 are omitted.
BE2018 / 5848
FIG. 24 is a schematic representation of a Z order of four windows or image planes 2481-2484 that can be used in the computer system 2300 of FIG. 23.
As can be understood from FIG. 14 and FIG. 15, provide embodiments with an overlay application with a perforated texture bitmap (illustrated in FIG. 21 and FIG. 22) for a transparency level T in the range of 0% to 100% on the one hand, and embodiments with an overlay application with a non-perforated texture bitmap (illustrated in FIG. 23 and FIG. 24) for a transparency level T in the range of 50% to 100% on the other hand, comparable (spatial average) results. These embodiments are extremely useful for converting a glossy (e.g., plastic-like) background surface to a matte (e.g., paper-like) background surface, which is more pleasant to read.
FIG. 25 shows the text fragment of FIG. 16, overlayed (covered) by an overlay image using a transparency level T of 50% (or an alpha value of 50%), wherein a minority portion of the overlay image is occupied by a blue line segment (as in FIG. 16) and a majority portion of the Overlay window comprises a perforated texture bitmap, the bitmap having 50% pure transparent pixels and the other 50% pixels forming a texture bitmap with an average color value of (R, G, B) = (128, 128, 128). The resulting overlayed image thus has an average color of approximately (224, 224, 224). Comparison of FIG. 25 and FIG. 18 provides an example of the difference between an overlay with a perforated monochrome bitmap, and an overlay with a perforated texture bitmap. This overlay image can be created by the computer system 1900 of FIG. 19.
FIG. 26 shows an example of an overlay image as can be generated by the computer system of FIG. 23, using a non-perforated texture bitmap with an average color of about (61, 61, 61) and using a transparency level of T = 80%, resulting in an average background color of the overlayed image of about (216, 216, 216). Comparison of FIG. 26 and FIG. 25 demonstrates that the use of a perforated bitmap does not, or does not significantly, impair the image quality, and therefore the readability of the textual information.
FIG. 27 shows the exemplary texture bitmap used to display the image of FIG. 25, before color adjustment, contrast adjustment and perforation. In this example, the texture bitmap was a grayscale image, but the invention is not limited thereto, and colored texture images can also be used.
BE2018 / 5848
Experiments have shown that the texture bitmap can easily be transformed into another texture bitmap with a desired color (Rnew, Gnew, Bnew) by treating each of the color components separately, for example using the following formulas or equivalent formulas:
Rnew [x, y] = minmax (0, CF x (Rorig [x, y] -Ravg) + Rnew, 255); [6]
Gnew [x, y] = minmax (0, CF x (Gorig [x, y] -Gavg) + Gnew, 255); [7]
Bnew [x, y] = minmax (0, CF x (Borig [x, y] -Bavg) + Bnew, 255); [8] where Rnew [x, y] is the Red component of the pixel of the new texture bitmap at position (x, y); and Rorig [x, y] is the Red component of the pixel of the original texture bitmap at position (x, y); and CF is the contrast factor (CF = 1 means no contrast increase, CF <1 means reducing the contrast, CF> 1 means increasing the contrast); Ravg is the average Red color component, Gavg is the average Green color component, and Bavg is the average Blue color component of the original texture bitmap; and where minmax (a, b, c) is a function that provides the result b limited to the range of a to c, thus returning the value b unless b <a in which case a is returned, or unless b> c in which case c is returned. CF is a floating point number, but the result of each formula is converted to an integer from 0 to 255 (also called a byte).
CF can be calculated using one of the following formulas, or an equivalent formula:
CF = K + L x T; [9] or
CF = M + N x T + Q x (TxT) [10] where K, L, M, N, Q are predefined constants that may depend on the specific texture bitmap chosen, and T is the transparency level of the overlay window.
Expressed in simple terms, these formulas can be used to shift a histogram of the pixel values of the texture bitmap up or down (e.g. from Ravg to Rnew), and which can broaden or narrow the width of the histogram characteristic.
FIG. 28 schematically illustrates how overlaying with a perforated bitmap, e.g., a perforated monochrome bitmap or a perforated texture bitmap, which is repeatedly shifted back and forth over a single pixel distance, can be used for time multiplexing or time averaging, in addition to the spatial means.
FIG. 28 (at time t1) shows a pattern consisting of four pixels, two of these pixels being fully transparent pixels P6, the other two pixels being color pixels that are opaque or that will be alpha blended. This 2x2 pixel pattern is typically repeated (tiled) over the entire overlay bitmap.
BE2018 / 5848
FIG. 28 (at time t2) shows a pattern similar to that of FIG. 28 (on time tl), but where other pixel locations are completely transparent.
It is interesting to note that the pattern of FIG. 28 (at t2) can be obtained by merely shifting the bitmap of FIG. 28 (on t1) with 1 pixel up, or 1 pixel to the left, or 1 pixel to the right or 1 pixel down. So if the overlay bitmap is deliberately chosen 1 pixel wider and / or 1 pixel higher than the width W and the height H of the overlay window, and if the overlay bitmap is shifted back and forth every frame with 1 pixel position, or every two frames, or every three frames, then the underlying image is overlayed in a time-multiplexed manner. If the multiplexing frequency is sufficiently high (for example, at least 15 Hz or at least 20 Hz or at least 25 Hz or at least 30 Hz), then time-multiplexing can reduce visual artifacts caused by overlaying with the perforated bitmap. If the multiplexing frequency is low (for example, less than 10 Hz), pixels on edges may seem to swirl.
It is also possible to use two separate bitmaps, each of W x H size, and to alternately alpha mix with one or the other bitmap, but such an implementation would require more memory and more time to create the two bitmaps, and is therefore not preferred.
FIG. 29 and FIG. 30 have been added for the sake of completeness to show that the principles described above still work, despite further image processing outside the overlay application itself. Experiments have shown that the screen resolution and the scale factor or zoom factor for text and applications selected in the calculator also influence how the final image will look. Two specific examples are shown here, in FIG. 29 (a) - (c) on the one hand and FIG. 30 (a) - (c) on the other hand.
FIG. 29 (a) shows a screenshot of the same image as used in FIG. 18 (with a perforated monochrome bitmap overlay), but with 2 lines, displayed on a display device with a native resolution of 3840 x 2160, configured at a resolution of 2560 x 1440, with a scaling factor (or zoom factor) for text and applications of 100% , such as can be selected in the screen resolution dialog in case of the Windows 10 operating system.
FIG. 29 (b) shows the image of FIG. 29 (a), zoomed in by a factor of approximately 350%. Individual pixels can be distinguished in this image. The checkerboard pattern becomes visible but the text remains very legible.
FIG. 29 (c) shows a portion of the image of FIG. 29 (b), further zoomed in by a factor of around 200%, so zoomed in total by a factor of around 700%. In this image the effect of overlaying with a perforated bitmap with 50% fully transparent pixels and 50% is
BE2018 / 5848 semi-transparent pixels clearly visible. It was very surprising to the inventors that the individual pixels are not, or not significantly, noticeable in FIG. 29 (a), even if they are actually present.
FIG. 30 (a) shows the same image as FIG. 29 (a) displayed on a display device with a native resolution of 3840 x 2160, configured at a resolution of 2560 x 1440, but with a scaling factor (or zoom factor) for text and applications of 125%, as can be selected, for example in the screen resolution dialog in case of the Windows 10 operating system.
FIG. 30 (b) shows the image of FIG. 30 (a), zoomed in by a factor of approximately 350%. Individual pixels can be distinguished in this image, but the checkerboard pattern is less pronounced.
FIG. 30 (c) shows a portion of the image of FIG. 30 (b), further zoomed in by a factor of around 200%, so zoomed in total by a factor of around 700%. In this image, the effect of overlaying with a perforated bitmap with 50% fully transparent pixels and 50% semi-transparent pixels is no longer clearly visible, but instead a 5 by 5 pixel pattern appears to have been added to the image, probably as a result of filtering related to zooming, performed outside the overlay application but within the computer device, since a screenshot can be made.
Again, the individual pixels are not, or not significantly, noticeable in FIG. 30 (a), even if they are actually present. The text of FIG. 30 (a) is very easy to read, and the line is easily distinguishable from the background.
It seems to be a matter of taste which of the two images, that of FIG. 29 (a) or that of FIG. 30 (a) is the best. The image of FIG. 29 (a) appears somewhat sharper, that of FIG. (A) appears to have some wrinkle or pattern due to the combined effect of the perforation followed by filtering, which is also aesthetically appealing, since it gives the impression of a matte background. In any case, both images provide text that is easy to read, both text in the background and text below the line, and in both cases the line is easily distinguishable from the background, and in both cases the average brightness of the white background is reduced, which can lead to less eye fatigue.
Experiments with other scaling factors for text and applications (in particular 150%,
175%, 225%, 250%, 350%) show results similar to those of FIG. 30 (a) to FIG. 30 (c), while scale factors of 200% or 300% appear to yield results similar to those of FIG. 29 (a) to FIG. 29 (c).
BE2018 / 5848
FIG. 31 shows a flow diagram of a computer-implemented method according to some embodiments of the present invention. This method is performed by a computer system of FIG. 9, FIG. 11, FIG. 19, FIG. 21, FIG. 36 and FIG. 38.
An overlay window is provided in step 3101.
The overlay window can be configured as an opaque window, which means that this window can have non-transparent pixels and / or fully transparent pixels, but no semi-transparent pixels.
Alternatively, the overlay window can be configured as a semi-transparent window with an alpha transparency α in the range of 1% to 99% or from 2% to 98% or from 5% to 95%, meaning that this window is completely transparent pixels or semi-transparent can have transparent pixels, but no opaque pixels.
At step 3102, at least one visible object is provided in said overlay window, the at least one visible object comprising a bitmap with a first plurality of pixels P6 that are fully transparent pixels, and a second plurality of pixels P7, P8 that are opaque pixels or semi -transparent pixels (so not completely transparent pixels); and wherein the first plurality of pixels P6 and the second plurality of pixels P7, P8 are interleaved (arranged alternately), for example arranged according to a checkerboard pattern.
In step 3103, the overlay window is configured in click-through mode.
Step 3103 can be performed before step 3102.
In one embodiment, the overlay window is configured as an opaque window, and the second plurality of pixels are P7, P8, opaque pixels.
In another embodiment, the overlay window is configured as a semi-transparent window, and the second plurality of pixels are semi-transparent pixels.
With configured in click-through mode it is meant that the operating system O / S has been informed that input messages (eg caused by mouse and keyboard events) must be sent to one or more applications that are overlayed, or to components thereof, instead of from to the overlay application itself. Or more correctly, if, for example, the overlay application contains two windows, one user interface window that is not configured in click-through mode and another window that is configured in click-through mode, the O / S will send mouse events to the user interface window when the mouse is above this window, and to the one or more underlying applications if the mouse is above the click-through window, but not above the user interface window.
FIG. 32 shows a flow diagram of a more specific method 3200 according to a preferred embodiment of the present invention. This method is performed by a computer system of FIG. 9, FIG. 11, FIG. 19, FIG. 36 and FIG. 38.
BE2018 / 5848
In step 3201, an overlay window is provided, and configured as a semi-transparent window with an alpha transparency α in the range of 1% to 99% or from 2% to 98% or from 5% to 95% or from 10% to 90% or from 20% to 80% or from 30% to 70%.
At step 3202, at least one visible object (e.g., a horizontal line) is provided in said overlay window that occupies only a minority portion (e.g., less than 20% or less than 10%) of the area of the overlay window. For example, if the overlay window has a width W and a height H, then the first object can be, for example, a short horizontal line with a length smaller than W / 2 and a height smaller than H / 10, and thus occupy an area smaller than WxH / 20; or may be, for example, a horizontal line extending over the entire width W of the overlay window, and having a height smaller than H / 10, thus occupying an area smaller than W x H / 10.
In step 3203, a bitmap, or an object with a bitmap, is provided, which bitmap has a first plurality of pixels P6 that are fully transparent, and a second plurality of pixels P7, P8 with a color value to be mixed with pixels of one or more underlying windows. The second plurality of pixels P7 can all have the same color (perforated monochrome bitmap), or they can have different colors. The colors can form a color gradient, or can form a texture bitmap. The first and second plurality of pixels are interleaved (alternately positioned), e.g. according to a fixed pattern, e.g. a checkerboard pattern. The bitmap can be above the visible object.
In step 3204, the overlay window is configured as a so-called click-through window. The operating system will send input events such as mouse clicks and mouse movements to one or more underlying windows (or to the desktop window).
In step 3205, position information X, Y is obtained from a so-called mouse pointer or mouse cursor, for example in the form of screen coordinates. The mouse cursor position can, for example, be requested from the operating system, for example in the case of the Windows O / S using the GetCursorPos function. It is an advantage to use these coordinates because this makes it possible to align or position the at least first visible object relative to the mouse cursor. Alternatively, motion information or movement information dx, dy can be obtained from the operating system in the form of raw input messages, for example by configuring the operating system to send raw input messages, and by configuring the overlay application to send raw input messages received and processed. This aspect is described in more detail in a co-pending patent application filed by the same applicant and on the same day as the present application, with approximately 72 claims and entitled METHOD AND DEVICE AND SYSTEM FOR PROVIDING DOUBLE MOUSE SUPPORT, which document is incorporated herein by reference in its entirety, and to which will be referred to herein as the co-in55
BE2018 / 5848 treatment being double mouse application. In the event of inconsistencies between the present application and the co-pending application, the present document prevails.
In step 3206, a position of the at least one visible object (e.g., the X and Y position of a short horizontal line, or the Y position of a large horizontal line) is adjusted in accordance with the obtained mouse cursor position X, Y or in accordance with the motion or displacement information dx, dy.
In the case of a small or large cross, the mouse cursor position is preferably at or near the intersection of the horizontal and vertical line. In the case of a small horizontal line, the mouse cursor may be at the left end or the right end or at the center of the horizontal line, or in any other suitable position.
Steps 3205 and 3206 are repeated, for example triggered by a timer event with a period in the range of 1 ms to 200 ms, for example in the range of 2 ms to 100 ms, for example equal to about 5 ms or about 10 ms or about 16.7 ms (60 Hz), or about 20 ms or about 25 ms or about 30 ms or about 40 ms or about 50 ms or about 100 ms or about 150 ms. This value is not critical. The smaller this value, the faster the overlay application responds to mouse movements, but the more computing power is required, among other things, for drawing the at least one object at intermediate positions during movement of the input device. The skilled person can find a suitable compromise between fast response time and a reasonable amount of computing power.
FIG. 33 shows a flow diagram of another method 3300 according to an embodiment of the present invention. This method is performed by the computer system of FIG. 7 and the computer system of FIG. 38.
An overlay window is provided in step 3301.
The overlay window may have a bitmap that substantially fills the entire area of the window with fully transparent pixels (as shown in FIG. 7 and in FIG. 38), or may include a perforated bitmap (such as in FIG. 10 or FIG. 12). ) or a perforated texture bitmap (as in FIG. 20 and FIG. 22) or a non-perforated texture bitmap (as in FIG. 24).
In case the window comprises fully transparent pixels (as shown in FIG. 7 and FIG. 38), the overlay window 883, 3983 can be configured as opaque or semi-transparent. In case the window comprises a perforated bitmap, the overlay window 883, 3983 is preferably configured as semi-transparent. Preferably the alpha transparency of the semi-transparent window is a value α in the range of 1% to 99% or from 2% to 98% or from 5% to 95% or from 10% to 90% or from 20% to 80 % or from 30% to 70%.
BE2018 / 5848
In step 3302, a first visible and movable object is provided in the overlay window that is in the form of a horizontal line 721, 3821 that occupies only a minority portion (e.g., less than 20% or less than 10%) of the area of the overlay window. The horizontal line preferably extends over the entire width of the overlay window, which preferably extends itself over the entire pixel area of the screen, or over the entire working area of the screen.
In step 3303, a second visible and movable object is provided in the overlay window that is in the form of a vertical line 722, 3822 that occupies only a minority portion (e.g., less than 20% or less than 10%) of the area of the overlay window. The vertical line preferably extends over the entire height of the overlay window, which preferably extends itself over the entire pixel area of the screen, or over the entire working area of the screen.
Together the horizontal line and the vertical line form a cross. If the window comprises a bitmap, the lines are positioned above the said bitmap so that they are visible. In the embodiment of FIG. 7, the horizontal line 721 and the vertical line 722 are monochrome lines, for example with a red or blue or black or purple or yellow color, or any other color. In this case, the overlay window is preferably configured as semi-transparent, so that text under lines 721, 722 is visible. In the embodiment of FIG. 38, the horizontal line 3821 and the vertical line 3822 contain a perforated bitmap. In this case, the overlay window 3983 can be configured as semi-transparent or it can contain fully transparent pixels.
In step 3304, the overlay window is configured as a so-called click-through window, so that the operating system will send keyboard and mouse events to the underlying window (s), even though the overlay window is above that, and has a higher Z-order.
In step 3305, position information X, Y is obtained from a so-called mouse pointer or mouse cursor, for example in the form of screen coordinates.
Alternatively, motion information related to movements of the pointing device can be obtained from the control system by using raw input messages, in which case the processing of the raw input messages and the actual update of the position of the visual object (and ) is preferably disconnected from each other, for example as explained in more detail in the pending double mouse request.
In step 3306, the position of the movable objects (e.g., the X coordinate of the vertical line, and the Y position of the vertical line) is adjusted according to the obtained mouse cursor position X, Y or according to the motion information.
The native mouse cursor position is preferably at or near the intersection of the horizontal and vertical lines.
Steps 3305 and 3306 are repeated, for example using a timer interrupt with a period in the range of 1 ms to 200 ms, for example in the range of 2 ms to 100 ms, for example equal to about 5 ms or about 10 ms, or approximately 16.7 ms (60 Hz), or
BE2018 / 5848 about 20 ms or about 30 ms or about 40 ms or about 50 ms or about 100 ms or about 150 ms.
FIG. 34 shows a simplified high-level block diagram as a possible representation of the software components and hardware components located in a computer system, which generally cooperate in carrying out a method according to embodiments of the present invention, but it is explicitly pointed out that the present invention does not belong to this implementation is limited, and that other implementations can also be used. The main purpose of this block diagram is to show a possible implementation of an embodiment implemented in a Windows-compatible computer.
FIG. 34 shows an overlay application 3474 and an operating system or O / S 3460 (e.g. MICROSOFT WINDOWS 10 available from Microsoft Corporation of Redmond, Wash, US), a graphics accelerator 3425, output interfaces 3480, a display 3492, an input interface 3410, an input pointing device 3401 and a bitmap generator 3420. Other components include, within the O / S 3460, input device drivers 3466 and graphical API (Application Programming Interface) 3440.
FIG. 34 is very similar to FIG. 8 of US 6333753 (B1), which is incorporated herein by reference in its entirety, but especially FIG. 8 thereof and the accompanying description. Since many aspects are already known, they do not need to be explained in full detail again.
The main similarities of the block diagram 3400 of this document and the block diagram of FIG. 8 of US6333753 (B1) are:
* the internal functioning of the video back-end components, the graphic API 3440, and the graphic accelerator 3425 and the output interfaces 3480 can be largely similar to those of the prior art, e.g. the way in which fully transparent pixels are processed , and the way alpha mixing is performed.
The major differences between the block diagram 3400 of this document and the block diagram of FIG. 8 of US6333753 (B1) are:
* the present invention does not require a mouse with touch sensors, and therefore does not require special device drivers (device drivers) 3466. Instead, conventional device drivers such as, for example, a standard USB mouse driver (mouse driver) can be used;
* the overlay application 3474 of the present invention does not use animated fadein and fade out. Optionally, however, time-multiplexing of the perforated bitmap can be used as described above (FIG. 28), in which case a time-multiplex component 3415 is used, which can send new position information of the offset perforated and / or textured bitmap, for example any frame, or every two frames, or every three frames, or every four frames. The timing is preferably synchronized with the frame rate;
BE2018 / 5848 * in the present invention, the bitmap generator 3420 can generate one or more stationary and / or movable objects, such as, for example, a small horizontal line, or a small cross (e.g. comprising a small horizontal line and a small vertical line), or a large horizontal line (which, for example, extends over the entire width of the overlay window), or a large cross (which, for example, comprises a large horizontal line that preferably extends over the entire width, and a large vertical line that extends preferably extends over the entire height);
* although not shown in FIG. 34, the present invention preferably also provides a user interface window, of which in FIG. 35 an example is shown. The user interface window is usually implemented as a second, substantially opaque window (it may comprise fully transparent pixels), and is preferably located (in Zorde) between the overlay window and the mouse cursor surface. Through this user interface window, the user can, for example, set or select or change dimensions of the horizontal line (pixels P3), and / or set or select the desired level of transparency T (or alpha transparency value α), and / or a color set or select monochrome pixels (P7) of the overlay bitmap, and / or set or select an average color of pixels of texture pixels (P8), etc. This user interface window is not configured in click-through mode because it must be able to click and / or keyboard input, and its area is generally reduced, e.g., minimized during normal use of the graphic overlay application.
Although not shown in FIG. 34, the computer device usually executes one or more application programs in addition to the overlay application 3474, such as, for example, a word processor, a PDF viewer, a web browser, a spreadsheet, etc.
The input positioning device 3401, for example a mouse or a touch pad, is connected via cables 3403 to input interface 3410, for example a USB input interface, which processes and / or forwards signals from the input device to input device drivers 3466 which component within the O / S 3460. The device drivers 3466 interpret the signals produced by the pointing device 3401 and generate response-relevant events. In particular, the O / S causes a mouse pointer or mouse cursor to move in accordance with movements of the device 3401, and transmits these events to an application (e.g., a text processor or web browser), since the overlay application 3474 is configured in click-through mode or pass mode.
Bitmap generator 3420 can include the following:
* a horizontal line generator 3422 for generating at least one horizontal line or beam with a predefined or customizable shape (e.g. rectangular with sharp edges or with round edges) and size (e.g. width and height) and position and view ( e.g. as a monochrome bitmap, or with a color gradient, or as a perforated bitmap);
BE2018 / 5848 * a vertical line generator 3424 for generating at least one vertical line or beam with a predefined or customizable shape (e.g. rectangular with sharp edges or with round edges) and size (e.g. width and height) and position and appearance (e.g., as a monochrome bitmap, or with a color gradient, or as a perforated bitmap);
* a background bitmap generator 3426 for generating a relatively large bitmap that preferably occupies the entire area of the overlay window, for example as a monochrome bitmap (similar to FIG. 6), or as a perforated bitmap with 50% fully transparent pixels P6 and 50% monochrome pixels P7 (as in FIG. 10 and FIG. 12), or as a perforated bitmap with 50% fully transparent pixels P6 and 50% texture pixels P8 (as in FIG. 20 and FIG. 22), or as a bitmap with 100% fully transparent pixels P5 (as in FIG. 8, FIG. 37 and FIG. 39), or as a bitmap with 100% textured pixels P8 (as in FIG. 24). One or more of these objects can be generated on the fly, optionally with the help of a Graphic API 3440, and / or with the help of a graphic accelerator 3425.
As explained above, in the case that a semi-transparent window is used as the (first) overlay window, the transparency level T thereof is preferably selected in the range of 30% to 75% (or the alpha-mixing value is preferably selected in the range of 25% to 70%), such that underlying document objects are clearly visible (although optionally slightly darkened or textured) through the overlay window.
In the case that time-division multiplexing is used (which is entirely optional), then the time-division multiplexing module or the time-division multiplexing process 3415 will issue instructions, preferably one such instruction for each N display frames (with N preferably being 1 or 2 or 3 or 4), to shift the perforated bitmap back and forth over a number of pixels, preferably an odd number of pixels, e.g. 1 pixel up / down or 1 pixel left / right, such that a particular pixel of the underlying document object alternately is overlayed (covered) by a completely transparent pixel or by a semi-transparent pixel. These instructions will be delivered, as schematically represented by line 3430 and / or line 3413, to O / S 3460, which in turn can pass these instructions on to Graphic API 3440 and ultimately to graphic accelerator 3425.
If no graphics accelerator is used, then, as symbolized by line 3455, Graphics API 3440 can deliver graphics output directly to output interfaces 3480 (specifically a standard video card, not shown, therein) which, in turn, will generate appropriate video signals and which provide signals via cables 3486 to display 3492. In this case, computer system 3400 will need to be sufficiently fast to implement the applicable graphics capabilities that would otherwise be provided by accelerator 3425 into software.
Although alpha-transparency functionality is supported by a wide variety of currently existing graphic accelerators, this functionality can easily be simulated in software, in a well-known way, by conventional 2-D (two-dimensional) or 3-D
BE2018 / 5848 (three-dimensional) graphics APIs, such as D3D (which is a 3-D graphics API, produced by Microsoft Corporation as a component of a WINDOWS operating system, where WINDOWS is a registered trademark of Microsoft Corporation), OpenGL (which currently available in the art) or GDI (which historically is only a 2-D low-level graphics processing layer, currently produced by Microsoft Corporation and also included as a standard component of a WINDOWS operating system).
In a variant of the block diagram of FIG. 34, the overlay application does not request the position of the native cursor position of the operating system, but further comprises a processor (handler) of raw input messages configured to process raw input messages provided by the operating system and adapted to the extracting motion information dx, dy from said event messages, and for moving at least one movable object, and optionally overruling the native cursor position of the O / S. Although not relevant to the understanding of the present invention, this aspect is described in more detail in the co-pending double mouse application.
The bitmap generator 3420 can generate one or more image data structures, which can be one or more bitmaps, or a set of commands in a language such as Display Postscript or Quickdraw. Regardless of the details, the image data structure must contain sufficient information so that it can be screened (if not already a bitmap) at the display resolution or otherwise processed for display on the display.
FIG. 35 shows an exemplary user interface window as can be used by an overlay application according to the present invention. This user interface window may be displayed as a second, opaque window, on top of the first overlay window 883, 1083, 1284, 2083, 2283, 2483, 3783, 3983, or may be displayed, for example, as a very small object (e.g., a rectangular object) with the name of the overlay application, and with a size smaller than 150 x 30 pixels) or can be minimized in the taskbar.
FIG. 36 shows a schematic block diagram of another computer system 3600 according to an embodiment of the present invention, with a computer device 3601 having an overlay application for providing an overlay window 3783 with a movable object 3610 having a perforated bitmap. The line 3610 is movable in accordance with movements of the input device 3603. This embodiment offers the advantage that the overlay window 3783 can be opaque, with most pixels being completely transparent pixels and yet the effect of a semi-transparent line 3610 can be obtained. Such an embodiment can
BE2018 / 5848 are particularly useful in computing devices that do not have alpha mixing functionality, such as, for example, eReader devices.
FIG. 37 is a schematic representation of a Z-order of four windows or image planes 3781 to 3784 that can be used in the computer system 3600 of FIG. 36.
FIG. 38 shows a schematic block diagram of another computer system 3700 according to an embodiment of the present invention, with a computer device 3801 having an overlay application for providing an overlay window 3983 with a movable object in the form of a perforated cross. The cross is composed of a horizontal line 3821 and a vertical line 3822, and is movable in accordance with movements of the input device 3803.
FIG. 39 is a schematic representation of a Z-order of four windows or image planes 3981-3984 that can be used in the computer system 3800 of FIG. 38.
FIG. 40 shows a schematic block diagram of another computer system according to an embodiment of the present invention, with an overlay application providing an overlay window 4184 with a movable object in the form of a horizontal line 4020, which preferably extends over the entire width of the screen.
The overlay window 4184 is preferably configured as a semi-transparent window, preferably with an alpha transparency in the range of 5% to 95%, or from 30% to 70%, although this is not absolutely required, and the window 4184 can also be configured as opaque, in which case the pixels P5 would be completely transparent pixels.
The overlay window 4184 comprises a semi-transparent line with pixels P3. An important advantage of this overlay application is that the line can be used to highlight or underline corresponding text fragments in two different applications, for example a PDF source text to be translated positioned on the left side of the screen 4004 and an MS-Word target text that contains the translated text, on the right side of the screen 4004. The line 4020 helps to quickly find the context again when switching back and forth between the two documents, for example during proofreading, thereby significantly increasing the cognitive burden of the proofreader reduced.
Since the overlay application is configured in click-through mode, the mouse 4099 can be used to click or drag or scroll anywhere on the screen. In particular, it makes it possible to move the PDF document up or down slightly (eg by dragging or dropping the
BE2018 / 5848) to keep the text fragments in it aligned with corresponding text fragments from the Word document.
The overlay application causes the Y coordinate of the line 4020 to change in accordance with the Y coordinate of the mouse cursor 4099. The Y coordinates may be identical, or may be slightly shifted, such that the line is slightly above or appears slightly below the mouse cursor.
FIG. 41 is a schematic representation of a Z order of five windows or image planes 4181-4185 that can be used in the computer system of FIG. 40.
The embodiment of FIG. 40 and FIG. 41 can be seen as a variant of the embodiment of FIG. 38 and FIG. 39 or as a variant of the embodiment of FIG. 7 and FIG. 8.
In a variant (not shown) of the embodiment of FIG. 40 and FIG. 41, the line 4020 contains a perforated bitmap (as in FIG. 37), or the line contains a color gradient (vertical or horizontal), or the line contains a texture bitmap.
FIG. 42 shows a schematic block diagram of another computer system 4200 according to an embodiment of the present invention, with an overlay application providing in an overlay window 4384 that includes a vertical line 4240 defining a left area and a right area of the screen 4204, and a horizontal line segment 4260 that extends across the width of either the left area or the right area.
The overlay window 4384 is preferably configured as a semi-transparent window, preferably with an alpha transparency in the range of 5% to 95%, or from 30% to 70%, although this is not absolutely required, and it can also be opaque be configured, in which case the pixels P5 would be completely transparent pixels.
The horizontal position of the vertical line 4240 may be predefined, e.g. substantially in the center of the overlay window 4384, or may be selectable via a user interface, but is preferably configurable by dragging the line with the mouse cursor 4299. The the latter is not evident if the overlay window 4384 is configured in click-through mode, but is possible by, for example, dynamically configuring the overlay window 4384 in click-through mode or not in click-through mode, depending on whether or not the mouse cursor is above vertical line 4240.
The overlay application is configured to update the vertical position Y of the horizontal line segment 4260 in accordance with the position of the mouse cursor 4299 or movements of the pointing device or position device, e.g. mouse device 4203, but only if the mouse cursor is in the left area (ie on the left side of the vertical line 4240).
BE2018 / 5848
When the mouse is in the right-hand area (i.e., on the right-hand side of the vertical line 4240), the horizontal line segment 4260 retains its position (which is also referred to herein as freezing).
An important advantage of this overlay application is that the line segment 4260 can be used to highlight or underline text fragments in a document to be translated (eg a PDF document located on the left side of the vertical line 4240) while working ( with the keyboard 4202 and the pointing device (e.g. mouse device 4203) in a text editor located on the right-hand side of the vertical line 4240. The line segment 4260 helps to quickly find the context again when switching back and forth between the two documents, for example during translation, thereby significantly reducing the translator's cognitive burden.
This overlay application can also be used by proofreaders, who use line segment 4260 to underline or highlight text fragments (by moving the mouse cursor to the left for each line or some lines of the source text), and then moving the mouse cursor to the right-hand area, and using the mouse cursor 4299 to hover over the target text, and optionally correcting the latter.
FIG. 43 is a schematic representation of a Z order of five windows or image planes 4381-4385 that can be used in the computer system of FIG. 42.
In a variant (not shown) of the embodiment of FIG. 42 and FIG. 43, the line segment 4260 contains a perforated bitmap (as in FIG. 37), or contains a color gradient (vertical or horizontal), or contains a texture bitmap.
In a variant of the embodiment of FIG. 42 and FIG. 43, the line segment 4260 is on the right-hand side of the vertical line 4240, and is movable if the mouse cursor or pointer 4299 is on the right of the vertical line 4240, and freezes if the mouse cursor or pointer is on the left-hand side of the vertical line 4240 . This embodiment is considered particularly suitable for left-handed persons.
FIG. 44 shows a flow diagram of a method performed by the computer system 4200 of FIG. 42 and FIG. 43 can be performed. FIG. 44 shows a computer-implemented method 4400 for overlaying a graphic image in a computing device 4201, the method comprising the steps of:
a) providing 4401 an overlay window 4384, preferably as a semi-transparent window;
b) providing 4402 a first visible object in the form of a vertical line 4240;
BE2018 / 5848
c) providing 4403 a second visible object in the form of a horizontal line segment 4260 which is located either on the left side or on the right side of the vertical line 4240;
repeatedly performing the following steps:
d) obtaining 4404 position information X, Y from a mouse pointer or mouse cursor 4299;
e) testing 4405 whether the mouse cursor or pointer 4299 is on or above the vertical line 4240, and if the result of the test is positive, proceed to step
f), otherwise continue with step h);
f) if the overlay window is not already configured in non-click-through mode, configuring 4406 of the overlay window 4384 in non-click-through mode (meaning not in click-through mode but in a mode where the operating system keyboard and mouse events to the overlay window or objects will send from it, instead of to underlying applications, if the mouse is positioned above the overlay window);
g) testing whether a mouse button is pressed, and if the result of this test is true, dragging the vertical line; and go back to step e);
h) if the overlay window is not already configured in click-through mode, configuring 4408 of the overlay window 4384 in click-through mode;
i) adjusting a position of the horizontal line segment 4260 if the mouse cursor or pointer 4299 is on the same side of the vertical line 4240 as the horizontal line segment 4260, and freezing the horizontal line segment 4260 if the mouse cursor 4299 is on the opposite side of the vertical line 4240, and return to step e).
This embodiment provides a line segment 4260 that freezes when the mouse cursor is moving on the right side of the vertical line 4240, and that moves along with the mouse cursor when the mouse cursor 4299 is on the left side of the vertical line 4240. Moreover, this embodiment makes it possible to drag the vertical line 4240 in a very intuitive manner. This embodiment is particularly suitable for translators and proofreaders who do not use a Computer Aided Translation tool (CAT) tool, or to verify a translation outside of the CAT tool.
It is noted that there was a requirement to make the vertical line position selectable or configurable, and since this configuration requires user interaction, the logical place to implement this functionality in the user interface window is because this window is capable of keyboard and receive mouse events. However, the inventors came up with the idea that it would be easier and more intuitive for the user if the vertical line could be dragged,
BE2018 / 5848 but that was not possible because the vertical line is implemented in a window that is configured in click-through mode. In an attempt to find a solution to make dragging possible, the inventors experimented with dynamically changing the click-through functionality of the overlay window, not knowing whether, let alone expecting it to work at all, and whether it would work reliably. to work. The experiments surprisingly demonstrated that it is indeed possible to dynamically change the click-through mode of a window depending on the position of the mouse cursor, in particular depending on whether the native mouse cursor is above the area of the vertical line. At least this aspect is not trivial.
In a variant of the method for FIG. 44, the line segment 4260 is on the right-hand side of the vertical line 4240, and is movable if the mouse cursor or pointer 4299 is on the right of the vertical line 4240, and freezes if the mouse cursor or pointer is on the left-hand side of the vertical line 4240 . This embodiment is considered to be especially useful for left-handed translators and proofreaders and writers.
FIG. 45 shows a mobile device such as, for example, an eReader or a smartphone device, which includes a touch screen, and which executes an app or application that displays textual information, e.g. in the form of an eBook, or a web browser. The textual information is overlayed (covered) by a line 4510 that allows a user to underline or highlight text information that he or she is reading. The line 4510 can be implemented in an overlay window associated with an overlay application or as part of the operating system running on the device, or can be implemented as part of the app or application itself, e.g. the eBook reader or web browser. browser app.
In one embodiment, the line 4510 is a semi-transparent line created in an overlay window with an alpha transparency in the range of 1% to 99% or from 2% to 98% or from 5% to 95%, preferably in the range of 30% to 70%. The line 4510 may contain a plurality of monochrome pixels (as in FIG. 8), or may include a color gradient (not shown), or may include a texture.
In another embodiment, the line 4510 is formed by, or includes, a bitmap comprising a first plurality of fully transparent pixels P6 and a second plurality of opaque or semi-transparent pixels P7 that are interleaved (alternately positioned) to make the line appear semi-transparent, referred to herein as a perforated bitmap. In case the pixels P7 are opaque pixels, no alpha mixing is required, and yet the line does not obscure the underlying textual information completely.
The line 4510 can be movable automatically, e.g. based on a timer. Preferably, the speed is configurable or adjustable via a user interface (not shown).
BE2018 / 5848
The line 4510 may be movable by dragging, in ways known per se in the prior art, based on a user who contacts the screen with his or her finger, and then moves the finger.
The line can help the reader stay focused while reading, despite distractions.
In an embodiment in which the line 4510 extends substantially the entire width of the display of the device 4500, the reader application or the overlay application need only adjust the vertical position (Y) of the line relative to the screen in accordance with user movements on the touch screen.
In an embodiment where the line 4510 extends only a fraction (e.g., less than 50%) of the width of the display of device 4500, the reader application or the overlay application may be adapted to have both a horizontal position X and a vertical adjust position Y of the line, in accordance with movements of a user's finger on the touch screen.
FIG. 46 shows a schematic block diagram of another computer system 4600 according to an embodiment of the present invention, which can be seen as a variant of FIG. 21, wherein the computer system 4600 provides a graphic overlay image 4784 in the form of a so-called perforated texture bitmap overlayed by the display device 4604.
In this embodiment, the computer device 4601 generates the image planes 4781 to 4783, and supplies corresponding image data to the display device 4604, as a conventional computer would, but the display device, instead of simply displaying said computer data on a display panel , overlays said image data with an overlay image 4784 stored in the display device, and displays the result.
Display devices are well known in the art, in particular stand-alone LCD display devices or stand-alone LCD monitors. Such a display device comprises essentially three parts: an application processor, a memory device that comprises a frame buffer, and a display device that comprises a display controller and a display panel. The application processor is responsible for drawing and generating an image frame, and for storing the generated image frame in the frame buffer in the memory device. The frame buffer is used to store data from pixels on a display screen. The display controller is primarily responsible for reading the data from the pixels from the frame buffer, and sequentially writing the data from the pixels to logic circuits on the display panel corresponding to the pixels. In a process of displaying an image by the display system, the application processor constantly generates a new image frame, and sends the image frame to the frame buffer.
BE2018 / 5848
In the context of the present invention, the application processor is adapted to provide an overlay image 4784 over said computer image, the overlay image being a perforated texture bitmap as explained above. In other words, this bitmap 4730 comprises or consists of a first plurality of pixels P6 that are fully transparent pixels, and a second plurality of pixels P8 that are semi-transparent pixels, which pixels P6 and P8 are interleaved (alternately positioned) in a regular pattern, preferably a checkerboard pattern. The actual step of overlaying may include alpha mixing the computer pixels with corresponding pixels of the perforated texture bitmap.
The perforated texture bitmap may be a predetermined bitmap built into the display device, or is preferably a downloadable image (e.g., displayed on the computer device, and captured by the display device upon user request, for example initiated by a print on a button of the display device (not shown), and then stored in a non-volatile memory of the display device for further use, but other means for downloading the texture bitmap have also been considered, for example using a USB interface for inserting a memory stick, or using a wireless connection, for example a Bluetooth connection for establishing a wireless communication with the computer device, in which case the computer device is preferably equipped with a special application for selecting and sending a texture bitmap to the display device.
Optionally, the display device is further adapted to scale and / or resize and / or tile and / or perforate the texture bitmap, although the perforation can also be implicitly implemented by only mixing a subset of the pixels, for example, one pixel for every two pixels according to the checkerboard pattern.
Preferably there are at least four or five alpha mixing levels, for example corresponding to a mix of 0/4 = 0%, 1/4 = 25%, 2/4 = 50%, 3/4 = 75% and 4/4 = 100 %. More preferably, there are at least eight or nine alpha mix levels, for example corresponding to a mix of 0/8 = 0%, 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7 / 8 and 8/8 = 100%.
Optionally, the display device is further adapted to adjust the transparency level and / or the color and / or the contrast of the texture bitmap, for example using some or all of the formulas [6] to [10] based on a predefined color and / or contrast level that may be provided to the display device by a user, using one or more buttons on the display device, or using said special application for configuring the display device using the aforementioned wireless connection, eg Bluetooth connection.
Optionally, the processor is further adapted to repeatedly adjust a position of the texture bitmap such that pixels of the underlying graphic image are overlayed
BE2018 / 5848 by a fully transparent pixel of the first plurality of pixels P6 at a first time point t1, and overlayed by a semi-transparent pixel P8 of the second plurality of pixels at a second and optionally third and fourth time points t2, t3, t4, for example, as explained above, when discussing FIG. 28.
Rather than physically shifting the texture bitmap back and forth over a horizontal or vertical distance of, for example, one pixel for each frame, and rather than storing a perforated bitmap, the processor may be adapted to combine the step of perforating simultaneously and time multiplexing in generating the image to be stored in the frame buffer, e.g. by proceeding as follows:
i) at even times (t2, t4) copying computer image data from the input buffer at pixel locations X, Y for which the sum of the row index X and the column index Y is odd, and by alpha mixing computer image data from the input buffer with corresponding pixel data from the texture bitmap at pixel locations X, Y for which the sum of the row index X and the column index Y is even; and ii) copying computer image data from the input buffer at pixel locations X, Y for which the sum of the row index X and the column index Y is even, and alpha mixing computer image data from the input buffer with odd times (t3, t5) corresponding pixel data from the texture bitmap at pixel locations X, Y for which the sum of the row index X and the column index Y is odd.
Although this solution requires a special display device, it provides several advantages: this solution can reduce eye fatigue for users, especially when the computer image contains a clear image. It is a further advantage of this solution that it can transform the appearance of a pure white image into a textured image, for example a paper-like image, that can be aesthetically more pleasant to read. It is an advantage that the display device performs alpha mixing instead of the CPU of the computer device, so the CPU load is not affected by this overlay. Finally, it is an advantage to use a perforated bitmap instead of a non-perforated bitmap because it requires less computing power, since only a subset of the pixels, for example, only 50% of the pixels of each frame, have to be mixed, while the other pixels without alpha mixing can be passed on.
FIG. 47 is a schematic representation of a Z order of three windows or image planes that can be used in the computer system of FIG. 46, and above, a fourth, semi-transparent overlay image is provided by the display device.
BE2018 / 5848
FIG. 48 shows a schematic block diagram of another computer system according to an embodiment of the present invention, which can be seen as a variant of FIG. 46, wherein the computer system provides a graphic overlay image in the form of an unperforated texture bitmap that is overlayed by the display device. All that is stated with regard to FIG. 46 and FIG. 47 also applies here, except that all pixels of the texture bitmap should be alpha mixed with the pixels of the computer image.
FIG. 49 is a schematic representation of a Z order of three windows or image planes that can be used in the computer system of FIG. 48, and above, a fourth, semi-transparent overlay image provided by the display device 4804.
AND LAST BUT NOT LEAST,
Although individual features have been illustrated in various drawings and in various embodiments of the present invention, it is contemplated that features of various embodiments may be combined as would be obvious to those skilled in the art upon reading this document.
Although all embodiments with a desktop computer and a mouse and a keyboard are shown, other computing devices and / or other input devices can also be used. For example, the mouse device can be replaced with other devices that provide the same or similar function and control, such as a pointing device, a track ball or a joystick or a touch pad or a stylus.
Although the invention has been described primarily for the Microsoft Corporation's Windows operating system, the invention is not limited thereto, and can also be used, for example, on an Apple computer with the Apple OS Macintosh OS, or on a smartphone, or on a eReader, or other devices with an operating system with a graphical user interface.
The present invention describes various embodiments, which can be summarized as follows:
1) a computer-implemented method of overlaying with a perforated bitmap;
This can be summarized as follows. A computer-implemented method for overlaying a graphic image in a computing device, comprising the steps of: a) providing an overlay window; b) providing at least one visible object in said overlay window, wherein the at least one visible object comprises a bitmap with a first plurality of pixels that are fully transparent pixels, and a second plurality of pixels that are opaque pixels or semi-transparent pixels ; wherein the first plurality of pixels P6 and the second plurality of pixels are interleaved (alternately positioned).
BE2018 / 5848 for example in a checkerboard pattern; c) configuring the overlay window in click-through mode. The object is movable in accordance with mouse movements.
2) a computer device for performing such a computer-implemented method,
3) a computer system comprising such a computer device,
4) a computer program product for carrying out such a method,
5) a computer-implemented method of overlaying with a large cross,
This can be summarized as follows. A computer-implemented method for overlaying a graphic image in a computing device, the method comprising the steps of: a) providing an overlay window; b) providing an elongated horizontally visible object; c) providing an elongated vertically visible object in said overlay window; d) configuring the overlay window in click-through mode; repeatedly f) obtaining position information X, Y from a mouse pointer or obtaining movement information dx, dy from the at least one pointing device; repeatedly g) adjusting a position of the first and second visible object based on the obtained position information or using the motion information. The crotch can be semi-transparent, and can extend over the entire height and width of the screen. Use of this overlay application as an add-on (extension) for spreadsheet application for marking alphanumeric information and / or for extracting information from tables.
6) a computer-implemented method of overlaying with a texture bitmap,
7) a computer-implemented method of overlaying with a vertical line to split the screen, and with a horizontal line that moves or freezes, whereby the vertical line can be dragged,
This can be summarized as follows. A computer-implemented method for overlaying a graphic image, comprising: a) providing an overlay window; b) providing a vertical line; c) providing a horizontal line located on the left or right side of the vertical line; repeatedly: d) obtaining position information from a mouse cursor; e) testing whether the mouse cursor or mouse pointer is on the vertical line, and if so, continue with step f), otherwise continue with step h); f) configuring the overlay window in non-click-through mode; g) testing whether a mouse button has been pressed and, if so, dragging the vertical line; and go to step d; h) configuring the overlay window in click-through mode; i) adjusting a position of the horizontal line. The vertical line splits the screen and can be dragged. The horizontal line moves or freezes, depending on which side the mouse cursor is moving.
8) a portable device (eg an eReader) with an overlay with a semi-transparent line,
BE2018 / 5848
This can be summarized as follows. A portable computing device comprising: a touch screen; at least one processing unit and a first memory for storing computer-executable instructions; wherein the instructions are configured to generate a graphic image containing textual information and to display that graphic image on the touch screen; wherein the instructions are further configured to generate a line or an elongated object that covers said textual information; wherein the line or the elongated object comprises a plurality of semi-transparent pixels with a transparency level of 5% to 95%, or wherein the line or the elongated object contains a first plurality of fully transparent pixels and a second plurality of opaque or semi-transparent pixels , which are interleaved (alternately positioned); wherein the instructions are further configured to detect a contact position on the touch screen, and to adjust a position of the line accordingly. For example an eReader device that displays textual information overlayed by a semi-transparent line.
9) a display device for overlaying with a perforated bitmap,
10) a display device with a movable object that looks semi-transparent due to time-multiplexed overlay,
It is explicitly pointed out that embodiments of the present invention can be combined in any suitable manner with embodiments of the co-pending double mouse request.
For example, by adding to the embodiments of the present invention a second pointing device and the associated functionality for obtaining double mouse support, as described in the co-pending application. Or, for example, by adding a special hub and / or a special display device and / or a special video adapter to embodiments of the present invention.
Some of these combined embodiments may be particularly suitable for translators, or proofreaders, or technical writers, or patent attorneys or other professional users who spend many hours in front of a computer screen and often switch back and forth between different documents because the cognitive burden is related the regaining of the current context after switching back and forth between documents can be considerably reduced thanks to the multiple visible objects that are movable through the two (or more) pointing devices, and because eye fatigue related to a bright background light can also be reduced.

权利要求:
Claims (15)
[1]
A computer-implemented method (3100) for overlaying a graphic image in a computing device, the method comprising the steps of:
a) providing (3101) an overlay window (1083; 1284; 2083; 2283; 3783; 3983), and configuring the overlay window as a semi-transparent window with an alpha transparency (α) in the range of 20% to 80% ;
b) providing (3102) at least a first visible object (930; 1130; 1930; 2130;
3710; 3821, 3822) in the overlay window, wherein the first visible object comprises a bitmap with a first plurality of pixels (P6) that are fully transparent pixels, and a second plurality of pixels (P7;
P8) which are semi-transparent pixels; and wherein the first plurality of pixels (P6) and the second plurality of pixels (P7; P8) are organized in a regular 2x2 pattern where exactly two of the four pixels are completely transparent, and wherein the two fully transparent pixels are diagonally opposite each other;
c) configuring (3103) the overlay window in click-through mode;
d) providing at least a second visible object (910; 1120; 1910; 3822) that is movable in accordance with movements of at least one pointing device (803; 1003; 1803; 3703), and that only has a minority portion of the area of the graphic image to be overlaid.
[2]
A computer-implemented method according to claim 1, wherein the first visible object (930; 1130; 1930; 2130) comprising the bitmap occupies a majority portion of the area of the graphic image to be overlaid.
[3]
A computer-implemented method according to any one of the preceding claims, further comprising the steps of:
- repeatedly performing the following step:
f) obtaining position information (X, Y) related to a position of a mouse pointer or mouse cursor; or obtaining motion information (dx, dy) related to a motion of the at least one pointing device;
- repeatedly performing the following step:
g) adjusting a position of the at least one object (3710; 3821, 3822) that is movable and / or of the at least one second object (910; 1120; 1910) that is movable, using said obtained position information (X, Y) or using the said movement information (dx, dy).
BE2018 / 5848
[4]
A computer-implemented method according to any one of the preceding claims, wherein the second object (910; 1120; 1910) that is movable has an elongated shape.
[5]
A computer-implemented method according to any of claims 1 to 3, wherein the at least one second movable object (3821, 3822) comprises a first movable element (3821) with an elongated shape that is horizontally oriented, and a second movable element (3822) having an elongated shape that is vertically oriented;
and wherein step g) comprises adjusting a position (Y) of the first movable element (3821) and adjusting a position (X) of the second movable element (3822).
[6]
A computer-implemented method according to claim 5, wherein the first movable element (3821) extends over an entire width of the overlay window, and wherein the second movable element (3822) extends over an entire height of the overlay window.
[7]
A computer-implemented method according to any one of the preceding claims, further comprising the step of:
time multiplexed overlay by repeatedly adjusting a position of the first object containing the bitmap or of the bitmap itself, such that pixels of the underlying graphic image are overlayed by a fully transparent pixel of the first plurality of pixels (P6) at a first time (t1), and being overlayed by an opaque or by a semi-transparent pixel (P7, P8) of the second plurality of pixels at a second and optionally third and fourth time point (t2, t3, t4).
[8]
A computer-implemented method according to any one of the preceding claims, wherein step b) comprises:
- providing a bitmap (1930; 2130; 2330) wherein the second group of pixels (P8) are extracted or derived from a texture bitmap.
[9]
A computer-implemented method according to claim 8, further comprising the step of:
- adjusting pixel values of the second plurality of pixels (P8) extracted or derived from said texture bitmap, as a function of an alpha transparency value (α) of the overlay window (2083; 2283; 2483).
BE2018 / 5848
[10]
A computer-implemented method according to claim 9, wherein the pixel values comprise three color components (R, G, B), and wherein each pixel value (R), (G), (B) is adjusted using a linear expression of the respective color component itself, and to limit the result of that linear expression to the range of 0 to 255.
[11]
A computer-implemented method according to any one of the preceding claims, wherein the graphic image is generated as a layered composition of a desktop image and one or more images associated with one or more applications;
and wherein the semi-transparent overlay window is created by an overlay application.
[12]
12. A computer device comprising:
at least one central processing unit (CPU), and a first memory connected to the at least one central processing unit (CPU) for storing computer-executable instructions therein;
- wherein the computer-executable instructions comprise code fragments for performing an overlay method according to any of claims 1 to 11.
[13]
A computer device according to claim 12, further comprising a graphic processing unit (GPU) with alpha mixing functionality, and a second memory connected to the graphic processing unit (GPU) for storing graphic information therein; and wherein the overlay code fragments are configured to use the graphics processor (GPU) to perform at least alpha mixing, and optionally also time multiplexing according to claim 7.
[14]
14. A computer system comprising:
- a computer device according to claim 12 or claim 13;
at least one pointing device connected to the computer device, wherein the pointing device is movable by a user, and wherein the computer-executable instructions are further configured to receive input data as an indication of movements of the pointing device;
at least one display device connected to an output of the computer device for displaying the graphic image mixed with the overlay image.
BE2018 / 5848
[15]
A computer program product for providing a graphic overlay, the computer program product including executable instructions that, when executed on at least one central processing unit (CPU) of a computer device according to claim 12 or 13, or a computer system according to claim 14, ensure that the
A computer device performs a method according to any one of claims 1 to 11.

类似技术:

---

公开号 | 公开日 | 专利标题

---

US7274377B2|2007-09-25|Viewport panning feedback system

---

CN112149382A|2020-12-29|Multi-window smart window placement using high DPI screens

---

DE202017105674U1|2018-03-20|Control a window using a touch-sensitive edge

---

BE1026358B1|2020-01-20|METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

---

US20210026508A1|2021-01-28|Method, device and computer program for overlaying a graphical image

---

BE1025952B1|2019-08-27|METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

---

WO2019063495A2|2019-04-04|Method, device and computer program for overlaying a graphical image

---

BE1026516B1|2020-03-09|METHOD, DEVICE AND COMPUTER PROGRAM FOR OVERLAYING A GRAPHIC IMAGE

---

NL2023600B1|2020-02-17|Method, device and computer program for overlaying a graphical image

---

Chegini et al.2017|Interaction Concepts for Collaborative Visual Analysis of Scatterplots on Large Vertically-Mounted High-Resolution Multi-Touch Displays.

---

BE1026977B1|2020-08-24|METHOD AND DEVICE AND SYSTEM FOR PROVIDING DOUBLE MOUSE SUPPORT

---

NL2022859B1|2020-10-08|WORKING METHOD AND EQUIPMENT AND SYSTEM WITH DOUBLE MOUSE SUPPORT

---

NL2022858B1|2020-10-08|WORKING METHOD AND EQUIPMENT AND SYSTEM WITH DOUBLE MOUSE SUPPORT

---

US20210405850A1|2021-12-30|Method, computing device and program code for facilitating information display on a display

---

EP3462335A1|2019-04-03|System and method for selecting a time stamp and generating user interface elements

---

EP3462336A1|2019-04-03|System and method for generating user interface elements with a viewing pane

---

JP5445646B2|2014-03-19|Display device

同族专利:

---

公开号 | 公开日

---

BE1025596A1|2019-04-24|

---

BE1025591A1|2019-04-24|

---

NL2021697B1|2019-04-08|

---

BE1025601B1|2019-04-29|

---

NL2021699B1|2019-04-08|

---

BE1026358A1|2020-01-14|

---

BE1025596B1|2019-04-29|

---

BE1025593B1|2019-04-29|

---

NL2021695B1|2019-04-08|

---

BE1025610B1|2019-04-29|

---

BE1025609A1|2019-04-25|

---

BE1025594A1|2019-04-24|

---

BE1025685A1|2019-06-03|

---

BE1025594B1|2019-04-29|

---

BE1025598B1|2019-04-29|

---

NL2021696B1|2019-04-08|

---

BE1025610A1|2019-04-25|

---

BE1025601A1|2019-04-24|

---

BE1025596A9|2019-05-06|

---

BE1025598A1|2019-04-24|

---

BE1025609B1|2019-04-29|

---

BE1025593A1|2019-04-24|

---

BE1025590A1|2019-04-24|

---

BE1025590B1|2019-04-29|

---

BE1025685B1|2019-06-11|

---

BE1025591B1|2019-04-29|

---

NL2021698B1|2019-04-08|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

US6118427A|1996-04-18|2000-09-12|Silicon Graphics, Inc.|Graphical user interface with optimal transparency thresholds for maximizing user performance and system efficiency|

---

US6597383B1|1997-08-25|2003-07-22|International Business Machines Corporation|Pointing apparatus and a pointing method|

---

US20130239067A1|2012-01-09|2013-09-12|International Business Machines Corporation|Fast cursor location|

---

US5283560A|1991-06-25|1994-02-01|Digital Equipment Corporation|Computer system and method for displaying images with superimposed partially transparent menus|

---

CA2076466C|1991-10-21|2002-01-15|Alan R. Tannenbaum|Method for buffering high bandwidth data from an input device|

---

US5442376A|1992-10-26|1995-08-15|International Business Machines Corporation|Handling multiple command recognition inputs in a multi-tasking graphical environment|

---

US5638501A|1993-05-10|1997-06-10|Apple Computer, Inc.|Method and apparatus for displaying an overlay image|

---

CA2124505C|1993-07-21|2000-01-04|William A. S. Buxton|User interface having simultaneously movable tools and cursor|

---

CA2124624C|1993-07-21|1999-07-13|Eric A. Bier|User interface having click-through tools that can be composed with other tools|

---

US5666499A|1995-08-04|1997-09-09|Silicon Graphics, Inc.|Clickaround tool-based graphical interface with two cursors|

---

GB2313682B|1996-05-29|2001-01-10|Icl Personal Systems Oy|Personal computer systems|

---

JPH1083253A|1996-09-10|1998-03-31|Hitachi Ltd|Information processor|

---

US6151030A|1998-05-27|2000-11-21|Intel Corporation|Method of creating transparent graphics|

---

US6333753B1|1998-09-14|2001-12-25|Microsoft Corporation|Technique for implementing an on-demand display widget through controlled fading initiated by user contact with a touch sensitive input device|

---

US6232957B1|1998-09-14|2001-05-15|Microsoft Corporation|Technique for implementing an on-demand tool glass for use in a desktop user interface|

---

US6509889B2|1998-12-03|2003-01-21|International Business Machines Corporation|Method and apparatus for enabling the adaptation of the input parameters for a computer system pointing device|

---

US6981229B1|2000-04-28|2005-12-27|Alias Systems Corp.|System for dynamically mapping input device movement as a user's viewpoint changes|

---

US20020167534A1|2001-05-10|2002-11-14|Garrett Burke|Reading aid for electronic text and displays|

---

US7620915B2|2004-02-13|2009-11-17|Ludwig Lester F|Electronic document editing employing multiple cursors|

---

US7802265B2|2004-03-15|2010-09-21|Imi Innovations, Inc.|Computer interface system using multiple independent graphical data input devices|

---

US7928959B1|2005-05-05|2011-04-19|Imi Innovations Inc.|System and method for multiple independent computer inputs using unique device and source identification systems|

---

JP2007180641A|2005-12-27|2007-07-12|Sony Corp|Imaging apparatus and display method of imaging apparatus|

---

US8360779B1|2006-12-11|2013-01-29|Joan Brennan|Method and apparatus for a reading focus card|

---

US8130194B1|2006-12-18|2012-03-06|Imi Innovations, Inc.|Non-mouse devices that function via mouse-like messages|

---

US9442648B2|2008-10-07|2016-09-13|Blackberry Limited|Portable electronic device and method of controlling same|

---

US20110035700A1|2009-08-05|2011-02-10|Brian Meaney|Multi-Operation User Interface Tool|

---

US8717289B2|2010-06-22|2014-05-06|Hsni Llc|System and method for integrating an electronic pointing device into digital image data|

---

KR101204186B1|2010-08-03|2012-11-23|정환희|Cursor displaying methode on display unit and recording medium having cursor displaying program|

---

US20120054672A1|2010-09-01|2012-03-01|Acta Consulting|Speed Reading and Reading Comprehension Systems for Electronic Devices|

---

US8866854B2|2011-10-31|2014-10-21|Microsoft Corporation|Consolidated orthogonal guide creation|

---

US9564085B2|2012-05-27|2017-02-07|Dialog Semiconductor Inc.|Selective dimming to reduce power of a light emitting display device|

---

US10915698B2|2013-12-31|2021-02-09|Barnes & Noble College Booksellers, Llc|Multi-purpose tool for interacting with paginated digital content|

---

KR102212562B1|2014-05-23|2021-02-08|삼성디스플레이 주식회사|Method of processing an image and an image processing device for performeing the same|

---

KR102253444B1|2014-07-08|2021-05-20|삼성디스플레이 주식회사|Apparatus, method, computer-readable recording medium of displaying inducement image for inducing eye blinking|

---

法律状态:
2020-03-02| FG| Patent granted|Effective date: 20200120 |

---

2020-08-27| MM| Lapsed because of non-payment of the annual fee|Effective date: 20191231 |

优先权:

---

申请号 | 申请日 | 专利标题

---

BE201705695|2017-09-29|

---

BE201705694|2017-09-29|

---